Compare commits

..
Author SHA1 Message Date
Jacob AlberandGitHub ef1e33619a Merge branch 'main' into copilot/fix-path-traversal-issue 2026-05-14 11:38:58 -04:00
Jacob AlberandGitHub 44bb9e9d1a Merge branch 'main' into copilot/fix-path-traversal-issue 2026-05-14 11:38:32 -04:00
e5644c4955 add E2E http test for checkpoint context id rejection
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/730258ef-2781-4a7d-b7cf-b5c40c11defc

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>
2026-05-14 15:36:19 +00:00
westeyandGitHub 3047ad3066 .NET: Harness console refactoring (#5811)
* Restructure harness console so that reactive app is the entry point

* Further refactoring to split tool formatters, improve UX, make console configurable and fix bugs

* Address PR comments.

* UX tweak

* Fix streaming text bug

* Address PR comments.
2026-05-14 15:22:11 +00:00
1d86b6a105 Clarify MSRC repro padding length in regression test
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/832f45a6-c01e-4da9-bf85-1ba7b5f302e6

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>
2026-05-14 15:19:11 +00:00
cafcc53483 Address PR review feedback: clarify URL-decode comment, isolate test root, add e2e workflow rejection tests
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/832f45a6-c01e-4da9-bf85-1ba7b5f302e6

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>
2026-05-14 15:18:13 +00:00
Evan MattsonandGitHub 0e12640c70 Improvements for DevUI (#5840) 2026-05-14 15:05:27 +00:00
Evan MattsonandGitHub ae666a4887 Python: Bump agent-framework-ag-ui to release candidate stage (#5844)
* Bump agent-framework-ag-ui to release candidate stage

* Mark agent-framework-ag-ui as rc in PACKAGE_STATUS
2026-05-14 14:56:34 +00:00
Jacob AlberandGitHub 9e18c60745 Merge branch 'main' into copilot/fix-path-traversal-issue 2026-05-14 10:45:46 -04:00
2b10c2a0bc Reject path-traversal context ids in foundry workflow checkpoint storage
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/fca3aae6-50eb-4726-8baf-2718217d4e79

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>
2026-05-14 13:55:16 +00:00
CopilotGitHublokitothcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Jacob Alber
eb40535436 .NET: Add Executor RouteBuilder Unit Tests (#5824)
* Add RouteBuilder unit tests

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/012f3b3b-acb9-4869-9084-b767cbe1885b

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

* Address RouteBuilder test review feedback

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/012f3b3b-acb9-4869-9084-b767cbe1885b

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

* Fix RouteBuilder test nullability warning

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/012f3b3b-acb9-4869-9084-b767cbe1885b

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

* Refine RouteBuilder test helpers

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/012f3b3b-acb9-4869-9084-b767cbe1885b

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

* Refactor overload int constants to HandlerOverload enum

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/19397f58-a88a-41cf-bd85-588f520e0d0f

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

* Fix ValueTask compatibility with .NET Framework 4.7.2

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/a8437809-0898-43a6-a950-09eb3417f58a

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

* Fix IDE0001 format errors - simplify generic type names

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/8573214e-ec42-4969-ba94-76bdc8ad3e59

Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>
Co-authored-by: Jacob Alber <jaalber@microsoft.com>
2026-05-14 13:36:38 +00:00
53 changed files with 3230 additions and 2315 deletions
-383
View File
@@ -1,383 +0,0 @@
# Workflows Unit Test Coverage — Incremental Plan
Companion to [`wf-coverage-summary.md`](./wf-coverage-summary.md). That document
is the snapshot; this document is the proposal for **how to close the gap**
between today's `Microsoft.Agents.AI.Workflows` coverage (79.7% line / 67.2%
branch) and the team target (85%) and personal stretch target (90%).
## Goals
1. Reach **≥85% line coverage** on `Microsoft.Agents.AI.Workflows`, with
branch coverage trending toward the same threshold.
2. Stretch to **≥90% line coverage**.
3. Prefer tests that exercise **public APIs** through their documented entry
points rather than tests that bind to internals.
4. Keep each PR **small and reviewable** (a single class or a small group of
tightly related classes), so coverage progresses incrementally rather than
in one large unreviewable change.
## Out of scope (covered by pending PRs — do not duplicate)
The following classes are already addressed by in-flight PRs and are
**explicitly excluded** from every wave below. Re-check them once the PR(s)
merge before adding any new tests.
- `RouteBuilder` — [#5824](https://github.com/microsoft/agent-framework/pull/5824)
- `WorkflowBuilderExtensions` (`ForwardMessage`, `ForwardExcept`, `AddChain`,
`AddExternalCall`, `AddSwitch`) and `SwitchBuilder`
[#5826](https://github.com/microsoft/agent-framework/pull/5826)
- `MagenticWorkflowBuilder`, `MagenticOrchestrator`, `MagenticTaskContext`,
and the Magentic event/state record types —
[#5833](https://github.com/microsoft/agent-framework/pull/5833)
- All of `Observability.*`, `OpenTelemetryWorkflowBuilderExtensions`, and the
related delivery-status / activity helpers — re-enabled by
[#5837](https://github.com/microsoft/agent-framework/pull/5837)
The following are also out of scope as not-worth-testing:
- `YieldsMessageAttribute`, `StreamsMessageAttribute` — both `[Obsolete]` and
ignored by the source generator and runtime per their `Obsolete` message.
## Method
Each wave below states:
- **Target classes** with current line coverage and the approximate uncovered-
line count from the snapshot.
- **What to test** — the user-visible behavior(s) the new tests should
exercise. Tests should go through the public API where possible (e.g.,
`WorkflowBuilder` / `WorkflowHostAgent` / `InProcessExecution`) and only
reach into internals when there is no public path.
- **Estimated line-coverage delta** — the *upper bound* assuming every
uncovered line in the listed classes becomes covered. Real deltas are
typically 6080% of the upper bound.
- **Suggested PR shape** — the smallest natural unit of work.
The waves are ordered by **(impact / effort)**, highest first. Stop after each
wave, re-run the snapshot, and re-prioritize.
---
## Wave 1 — Public API holes that are easy to write (largest single jump)
These are public types where the missing tests are mechanical (constructors,
guards, equality, simple round-trips) but cumulatively account for ~140
uncovered lines.
### 1A. `Workflow` (35 uncov, 44.4% line / 44.1% branch) — **public**
The bulk of the gap is around ownership and protocol-description paths that
are reachable but never asserted on directly:
- `Workflow.TakeOwnership` / `ReleaseOwnershipAsync`: each of the four
`(subworkflow, _ownedAsSubworkflow)` switch arms (the four error messages
enumerated in `TakeOwnership`), the "release by non-owner" branch, the
`ObjectDisposedException`-substitute branch, and the
`IResettableExecutor`-failure branch (the `"Cannot reuse Workflow with shared
Executor instances that do not implement IResettableExecutor."` path).
- `Workflow.DescribeProtocolAsync`: protocol described from a workflow whose
start executor is also an output executor; protocol described from a
workflow whose start executor binds to a `RequestPort`.
- `Workflow.ReflectPorts` / `ReflectExecutors` / `ReflectEdges`: returns
fresh-copy semantics (mutating returned dictionary does not mutate
`Workflow`).
**Suggested PR**: `WorkflowOwnershipAndReflectionTests.cs`. Covers ownership
state machine + the three `Reflect*` accessors + `DescribeProtocolAsync`.
### 1B. `EdgeId` (15 uncov, 37.5% / 0%) — **public struct**
Zero branch coverage on a public equality type. Add a single
`EdgeIdEqualityTests.cs`:
- `Equals(object)` against `null`, `EdgeId`, `int`, and an unrelated type.
- `Equals(EdgeId)`.
- `==` and `!=` operators (equal, unequal, default).
- `GetHashCode()` consistency with `Equals`.
- `ToString()` round-trip with the underlying index.
This is ~10 minutes of code, lifts a public struct from 37→100%, and
contributes to the branch-coverage number disproportionately.
### 1C. `Execution.EdgeConnection` (30 uncov, 36.1% / 37.5%) — **public**
Public class with a documented ID-uniqueness factory and an `IEquatable<>`
contract that is largely untested. Add `EdgeConnectionTests.cs`:
- `EdgeConnection(sourceIds, sinkIds)` null-guards on both arguments.
- `EdgeConnection.WithUniqueIds(...)` (the factory documented in the source
comment as enforcing uniqueness): both sides must reject duplicates with
`ArgumentException`.
- `Equals` / `GetHashCode`: equal when both ordered lists match; unequal when
source order differs (ordering is documented as significant); unequal when
sink order differs.
- `ToString` / display formatting if applicable.
### 1D. `EdgeId` and `EdgeConnection` together unlock `EdgeIdConverter`
Once 1B + 1C land, `EdgeIdConverter` (currently 80%) typically reaches 100%
without additional code because the new equality assertions cause the
serializer round-trip to be exercised by existing JSON tests.
**Wave 1 upper-bound delta:** ~80 lines (~1.5 percentage points).
---
## Wave 2 — `WorkflowBuilder` public-surface gaps (largest single class)
`WorkflowBuilder` is currently 82.4% line / 79.4% branch with **48 uncovered
lines** — by far the largest absolute gap on a public type that is **not**
already addressed by [#5824](https://github.com/microsoft/agent-framework/pull/5824)
or [#5826](https://github.com/microsoft/agent-framework/pull/5826).
Audit the file (`dotnet/src/Microsoft.Agents.AI.Workflows/WorkflowBuilder.cs`)
for the specific uncovered lines before writing tests; the report-generator
HTML output highlights them. From the snapshot, the under-tested behaviors
are:
- The `WithName` / `WithDescription` / `WithStartExecutor` chain on
pre-existing builders (re-assigning a start executor; assigning to the same
executor twice).
- `AddFanInEdge` validation paths (duplicate sources, source equals sink,
empty source list).
- `Build()` validation: workflow with no edges, workflow whose start executor
is unreachable from any registered executor (cycle-only graph), workflow
with a `RequestPort` not associated with any executor.
- `BindFunctionExecutor` / `Bind` overloads that take a custom id collision —
the duplicate-id branch.
- The "telemetry context already set" branch reached when
`OpenTelemetryWorkflowBuilderExtensions.WithOpenTelemetry` is called twice
on the same builder.
**Suggested PR**: `WorkflowBuilderValidationTests.cs`, focused on the
not-yet-exercised validation branches and the rebinding behaviors. Avoid
duplicating anything already in `WorkflowBuilderSmokeTests.cs`.
**Wave 2 upper-bound delta:** ~48 lines (~0.9 pp).
---
## Wave 3 — `ExecutorBindingExtensions` (public, 32% covered)
`public static class ExecutorBindingExtensions` (50 lines, 34 uncovered) is
the canonical entry point for turning user code into `ExecutorBinding`
instances. Many of its overloads are completely untested:
- `BindExecutor(this Executor)` — round-trip through `WorkflowBuilder` to
prove the `ExecutorInstanceBinding` resolves to the supplied instance.
- `BindExecutorFactory<TExecutor>(Func<ValueTask<TExecutor>>)` — including
the eager-instantiation behavior documented in the XML comment ("...will be
instantiated if a `ProtocolDescriptor` for the `Workflow` is requested, and
it is the starting executor.").
- `BindExecutorFactory<TExecutor>(string id, Func<ValueTask<TExecutor>>)`
custom-id overload.
- `BindFunction` / `Bind<TInput>(Func<…>)` / `Bind<TInput, TOutput>(Func<…>)`
— both sync and async overloads, both with and without `CancellationToken`,
including null-guard on the function and on the id.
**Suggested PR**: `ExecutorBindingExtensionsTests.cs`. Mostly mechanical;
each overload becomes one or two `[Theory]` rows. This single PR alone is
worth ~30 lines / ~0.6 pp on its own and removes a 32%-covered public type
from the lagging list.
---
## Wave 4 — `WorkflowSession` and `WorkflowHostAgent` (public hosting surface)
`WorkflowSession` is internal but is the implementation of the
**public** `WorkflowHostAgent`. Together they account for ~80 uncovered
lines and several documented public guarantees that have no test coverage
today.
The uncovered lines concentrate in three areas:
1. **Resume from a serialized session.** `WorkflowSession(Workflow, JsonElement,
IWorkflowExecutionEnvironment, …)` — currently exercised only on the happy
path. Add tests for: invalid JSON payload, payload missing pending-request
ids, payload referencing a different `Workflow` shape, round-trip through
`Serialize()` → re-construct.
2. **External-request reconciliation.** The `_pendingRequests` plumbing
(`AddPendingRequest`, `RemovePendingRequest`, `TryGetPendingRequest`) and
the `NormalizeResponseContentForDelivery` / `CreateRequestContentForDelivery`
/ `CloneFunctionCallContent` / `CloneFunctionResultContent` /
`CloneToolApprovalRequestContent` / `CloneToolApprovalResponseContent`
helpers. Each helper has at least one branch (id mismatch, missing
metadata, unsupported `AIContent` subtype) that is not hit today.
3. **`WorkflowHostAgent` failure paths.** Calling the agent against a
`Workflow` with no checkpointing configured when checkpointing is required
(the path validated by `VerifyCheckpointingConfiguration`); calling
`RunAsync` after the session has terminated; passing `null` for the
message list.
**Suggested PRs**:
- 4a — `WorkflowSessionResumeTests.cs` (resume + serialize round-trip).
- 4b — `WorkflowSessionExternalRequestTests.cs` (request/response cloning and
pending-request bookkeeping).
- 4c — `WorkflowHostAgentValidationTests.cs` (agent-level guards).
**Wave 4 upper-bound delta:** ~80 lines (~1.5 pp). This is the single largest
uncovered area on the `Workflow*` public surface.
---
## Wave 5 — Checkpointing converters and the file-system store
Cluster of small but high-percentage wins. None of these is glamorous, but
together they take the `Checkpointing.*` namespace from "spotty" to
"comprehensive" and remove every <60% type from the namespace except the
abstract `JsonCheckpointStore` (which by itself is 1 line and not worth a
dedicated test).
| Class | Cov | Uncov | What's missing |
| ----- | ---:| ----:| -------------- |
| `Checkpointing.ExecutorIdentityConverter` | 12.5% | 14 | Read/write paths for null id, mixed-case id, non-string token. |
| `Checkpointing.JsonWireSerializedValue` | 48.1% | 14 | Read with missing `TypeId`, missing `Value`, type-id mismatch on `As<T>()`. |
| `Checkpointing.PortableValueConverter` | 57.5% | 14 | Round-trip of `null`, primitive, complex object, and the unknown-type branch. |
| `Checkpointing.FileSystemJsonCheckpointStore` | 62.2% | 20 | Index-file rebuild on missing index, corrupted index file (existing test only covers the happy path), retrieval of unknown checkpoint id, store created against a path that does not yet exist. |
| `Checkpointing.WorkflowInfo` | 80.3% | 10 | Equality/serialization branches for workflows that vary only in `Name` / `Description`. |
`FileSystemJsonCheckpointStoreTests.cs` already exists; the additions can go
into the same file.
**Suggested PRs**: one PR per file, or one combined "checkpointing-converter"
PR plus a separate "FileSystemJsonCheckpointStore-edge-cases" PR.
**Wave 5 upper-bound delta:** ~70 lines (~1.3 pp).
---
## Wave 6 — `HandoffWorkflowBuilderCore<TBuilder>` (public base, 80.7% / 65.7%)
`HandoffWorkflowBuilderCore` is the public base used by `HandoffWorkflowBuilder`
and friends. With 22 uncovered lines and only 65.7% branch coverage on a
public surface, it is one of the two remaining public builders without
exhaustive validation tests (`WorkflowBuilder` itself is the other — see
Wave 2). Cross-check against existing `HandoffOrchestrationTests.cs` and
`HandoffMessageFilterTests.cs` to avoid duplication.
Behaviors to cover:
- Null/empty arguments to each public `Add*` / `WithStart*` / `WithEnd*`
method.
- Duplicate-id detection (per the `WorkflowBuilder` contract).
- `Build()` failure when no end executor is wired up.
- `Build()` failure when no handoff filter is supplied and the default rejects
the configured handoff target.
**Wave 6 upper-bound delta:** ~22 lines (~0.4 pp).
---
## Wave 7 — Public statics and remaining 0% public types
These are short tests with disproportionate impact on the *count of public
types with 0% coverage*, which is itself a useful quality metric independent
of the line-coverage number.
- **`StatefulExecutor<TState, TInput, TOutput>` (public, 0% / 9 lines)** —
add one minimal subclass that returns a value from `HandleAsync`, drive it
through `WorkflowBuilder` + `InProcessExecution`, assert state survives
across handler invocations. Mirror the existing tests for the two-type
`StatefulExecutor<TState, TInput>` overload (which is already covered).
- **`Specialized.RequestPortExtensions` (internal, 7.1% / 13 lines)** — three
tests: `ShouldProcessResponse` returns `false` when the response targets a
different port; throws `InvalidOperationException` (with the specific
message produced by `CreateExceptionForType`) when port matches but the
payload type does not; returns `true` on a matching port and matching
payload.
- **`MagenticPlanReviewRequest` / `MagenticPlanReviewResponse` (public, 0%)** —
PR [#5833](https://github.com/microsoft/agent-framework/pull/5833) covers
`MagenticPlanReviewResponse`. After it merges, add a one-shot test that
constructs `MagenticPlanReviewRequest` with each documented constructor
argument combination and asserts the property values, only if it is still
not covered.
- **`WorkflowEvaluationExtensions` (public, 89.4%, 12 uncov)** — the gap is
in two paths: the cancellation path through `EvaluateAsync` and the
evaluator-throws path. Two `[Fact]`s.
**Wave 7 upper-bound delta:** ~35 lines (~0.6 pp).
---
## Wave 8 — Internal helpers behind public hot paths
Lower priority than waves 17 because the public-API gaps should be closed
first. Listed here so they aren't forgotten when chasing the 90% stretch
target.
- `Execution.ExecutorIdentity` (15 uncov) — equality with `null`, with
`string`, with mismatched-case `ExecutorIdentity`, implicit `string` ↔
`ExecutorIdentity` conversions.
- `InProc.InProcStepTracer` (20 uncov, but only 21.4% **branch** coverage) —
drive a workflow that has both internal and external messages in the same
super-step; assert `SuperStepStartInfo.HasExternalMessages`,
`SuperStepCompletionInfo.HasPendingMessages` /
`HasPendingRequests`, the `Reload(lastStepNumber)` API, and the
`ToString()` formatting.
- `MessageMerger` (13 uncov, 89.6% / 85.7%) — the small remaining branches
in `MessageMerger.ResponseMergeState` (split across multiple final
fragments) and the empty-input branch.
- `ExecutorBinding` (11 uncov) — the abstract base; covered automatically as
Waves 3 and 4 land. Re-check after.
- `Visualization.WorkflowVisualizer` (18 uncov, 92.1%) — the remaining gaps
are escape-handling for unusual executor labels (newlines, double-quotes,
unicode) and the "subworkflow-with-zero-edges" formatting path. Three
`[Theory]` rows.
**Wave 8 upper-bound delta:** ~75 lines (~1.4 pp).
---
## Cumulative projection
| After… | Upper-bound line cov | Realistic (≈70%) |
| ------ | ---: | ---: |
| Today | 79.7% | — |
| In-flight PRs (#5824, #5826, #5833, #5837) merged | ~8687% | — |
| Wave 1 (public structs + `Workflow` ownership) | +1.5 pp | +1.0 pp |
| Wave 2 (`WorkflowBuilder` validation) | +0.9 pp | +0.6 pp |
| Wave 3 (`ExecutorBindingExtensions`) | +0.6 pp | +0.4 pp |
| Wave 4 (`WorkflowSession` + `WorkflowHostAgent`) | +1.5 pp | +1.0 pp |
| Wave 5 (checkpointing converters + file-system store edges) | +1.3 pp | +0.9 pp |
| Wave 6 (`HandoffWorkflowBuilderCore`) | +0.4 pp | +0.3 pp |
| Wave 7 (zero-coverage public types) | +0.6 pp | +0.4 pp |
| Wave 8 (internal helpers) | +1.4 pp | +1.0 pp |
Realistic projection after **all four pending PRs + Waves 14**: **~8990%
line coverage**, comfortably past the 85% team target and reaching the 90%
stretch target. Waves 58 then take the assembly above 90% on both line and
branch coverage.
## Workflow for executing the plan
1. Wait for the four in-flight PRs (#5824, #5826, #5833, #5837) to merge.
2. Re-run the snapshot in
[`wf-coverage-summary.md`](./wf-coverage-summary.md) and refresh the
per-class table. Some entries in this plan may already be addressed; drop
them.
3. Open Wave 1 as **one** PR per sub-section (1A, 1B, 1C). Keep each PR
focused on a single class so review stays mechanical.
4. After each wave merges, re-run the snapshot and re-prioritize. If a class
listed in a later wave is already past 90% coverage, drop it.
5. When the assembly clears 85% line coverage, raise the
`dotnet-check-coverage.ps1` threshold and add
`Microsoft.Agents.AI.Workflows` to the `nonExperimentalAssemblies` list in
that script so the bar cannot regress.
## Conventions for the new tests
- Follow the existing test-project conventions in
`dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests`:
xUnit v3 + Microsoft Testing Platform, `FluentAssertions`, and the existing
helper classes in `TestingExecutor.cs`, `TestRunContext.cs`,
`TestWorkflowContext.cs`, `MessageDeliveryValidation.cs`, etc. Reuse them
rather than introducing parallel infrastructure.
- Preserve the `Throw.IfXYZ` validation idiom in production code. New tests
should assert against the existing `ArgumentNullException` /
`ArgumentException` thrown by those helpers; do not change production
validation to a different style.
- For each new test file, prefer `[Theory]` over many near-duplicate
`[Fact]`s when only a single argument varies — this keeps reviewer load
low and matches the convention used in `WorkflowBuilderSmokeTests.cs` and
`RouteBuilderTests.cs` (PR #5824).
@@ -1,156 +0,0 @@
# Workflows Unit Test Coverage — Snapshot
This document captures a coverage snapshot of `Microsoft.Agents.AI.Workflows`
produced by running the existing `Microsoft.Agents.AI.Workflows.UnitTests`
suite. It is the input for [`wf-coverage-plan.md`](./wf-coverage-plan.md), which
proposes incremental work to lift coverage to the 85%/90% targets.
## How this snapshot was produced
```bash
cd dotnet
dotnet build tests/Microsoft.Agents.AI.Workflows.UnitTests/Microsoft.Agents.AI.Workflows.UnitTests.csproj -f net10.0
dotnet test --project tests/Microsoft.Agents.AI.Workflows.UnitTests/Microsoft.Agents.AI.Workflows.UnitTests.csproj \
-f net10.0 --no-build \
--coverage --coverage-output-format cobertura \
--coverage-output coverage.cobertura.xml \
--coverage-settings tests/coverage.runsettings
reportgenerator \
-reports:tests/Microsoft.Agents.AI.Workflows.UnitTests/bin/Debug/net10.0/TestResults/coverage.cobertura.xml \
-targetdir:./coverage-report \
-reporttypes:'Html_Dark;JsonSummary;TextSummary' \
-assemblyfilters:'+Microsoft.Agents.AI.Workflows;-Microsoft.Agents.AI.Workflows.Declarative*'
```
- Tooling: `dotnet test` with `Microsoft.Testing.Extensions.CodeCoverage` (the
same configuration documented in `dotnet/.github/skills/build-and-test/SKILL.md`).
- Scope: only the `Microsoft.Agents.AI.Workflows` assembly. The
`Microsoft.Agents.AI.Workflows.Declarative*` family is excluded — those have
their own test projects and a different ownership area.
- Test result: **460 passed, 22 skipped, 0 failed** (482 total). The 22 skipped
tests are the previously-flaky `ObservabilityTests` and
`WorkflowRunActivityStopTests`; **PR [#5837](https://github.com/microsoft/agent-framework/pull/5837)
re-enables them all.**
## Headline numbers
| Metric | Value | Target |
| --------------------- | --------------------------- | ------ |
| Line coverage | **79.7%** (4386 / 5499) | 85% / 90% |
| Branch coverage | **67.2%** (1111 / 1653) | — |
| Method coverage | **80.9%** (903 / 1115) | — |
| Fully-covered methods | **70.8%** (790 / 1115) | — |
| Coverable lines | 5,499 | — |
| Uncovered lines | **1,113** | — |
| Classes | 237 (185 source files) | — |
Distance to the 85% line target ≈ **+292 covered lines**.
Distance to the 90% personal target ≈ **+567 covered lines**.
The much larger branch-coverage gap (67.2%) is the bigger structural problem:
many code paths are reached but not all branches inside them are exercised.
## In-flight PRs that already address part of this gap
The plan in [`wf-coverage-plan.md`](./wf-coverage-plan.md) **excludes** the
classes covered by these PRs to avoid duplicate work or merge conflicts:
| PR | Classes added/improved |
| -- | ---------------------- |
| [#5824](https://github.com/microsoft/agent-framework/pull/5824) — `RouteBuilder` unit tests | `RouteBuilder` (143 lines, 58 currently uncovered) |
| [#5826](https://github.com/microsoft/agent-framework/pull/5826) — `WorkflowBuilder` specialized edge tests | `WorkflowBuilderExtensions` (51 lines, 21 uncov), `SwitchBuilder` (already 90.4%, but null-guard branches added) |
| [#5833](https://github.com/microsoft/agent-framework/pull/5833) — Magentic E2E coverage | `MagenticWorkflowBuilder` (69 lines, 0%), `MagenticOrchestrator` (52%), `MagenticTaskContext` (35%), and the related Magentic event types (all 0%) |
| [#5837](https://github.com/microsoft/agent-framework/pull/5837) — Re-enable `ObservabilityTests`/`WorkflowRunActivityStopTests` | `Observability/WorkflowTelemetryContext` (42.7%), `Observability/ActivityExtensions` (0%), `Observability/EdgeRunnerDeliveryStatusExtensions` (0%), `OpenTelemetryWorkflowBuilderExtensions` (0%), and the bulk of `Tags` / `EventNames` / `ActivityNames` |
Estimated combined impact once all four merge: **~360410 of the current 1,113
uncovered lines** become covered, lifting line coverage to roughly **8687%**
even before any of the new work proposed in the plan.
## Top uncovered classes (after excluding in-flight PRs)
Sorted by uncovered line count. "Public surface" indicates whether the class
itself or its uncovered members are part of the public API of the
`Microsoft.Agents.AI.Workflows` assembly — those are the highest-leverage
targets.
| # | Class | Public surface | Line cov | Branch cov | Coverable | Uncov |
|---|-------|:-:|---:|---:|---:|---:|
| 1 | `WorkflowSession` | internal (reached via `WorkflowHostAgent`) | 67.7% | 47.5% | 217 | **70** |
| 2 | `WorkflowBuilder` | **public** | 82.4% | 79.4% | 273 | **48** |
| 3 | `Workflow` | **public** | 44.4% | 44.1% | 63 | **35** |
| 4 | `ExecutorBindingExtensions` | **public** | 32.0% | n/a | 50 | **34** |
| 5 | `Execution.EdgeConnection` | **public** | 36.1% | 37.5% | 47 | **30** |
| 6 | `HandoffWorkflowBuilderCore<TBuilder>` | **public (base)** | 80.7% | 65.7% | 114 | **22** |
| 7 | `Checkpointing.FileSystemJsonCheckpointStore` | **public** | 62.2% | 57.1% | 53 | **20** |
| 8 | `InProc.InProcStepTracer` | internal | 62.9% | 21.4% | 54 | **20** |
| 9 | `Visualization.WorkflowVisualizer` | **public** | 92.1% | 93.5% | 230 | **18** |
| 10 | `EdgeId` | **public** struct | 37.5% | 0.0% | 24 | **15** |
| 11 | `Execution.ExecutorIdentity` | internal | 28.5% | 18.7% | 21 | **15** |
| 12 | `Checkpointing.ExecutorIdentityConverter` | internal | 12.5% | 0.0% | 16 | **14** |
| 13 | `Checkpointing.JsonWireSerializedValue` | internal | 48.1% | 50.0% | 27 | **14** |
| 14 | `Checkpointing.PortableValueConverter` | internal | 57.5% | 35.7% | 33 | **14** |
| 15 | `Execution.StateScope` | internal | 79.0% | 72.2% | 62 | **13** |
| 16 | `MessageMerger` | internal | 89.6% | 85.7% | 126 | **13** |
| 17 | `Specialized.RequestPortExtensions` | internal | 7.1% | 0.0% | 14 | **13** |
| 18 | `WorkflowEvaluationExtensions` | **public** | 89.4% | 75.0% | 114 | **12** |
| 19 | `ExecutorBinding` | internal (base) | 60.7% | 28.5% | 28 | **11** |
| 20 | `Checkpointing.WorkflowInfo` | internal | 80.3% | 54.5% | 51 | **10** |
| 21 | `WorkflowHostAgent` | **public** | 70.5% | 75.0% | 34 | **10** |
| 22 | `StatefulExecutor<TState, TInput, TOutput>` | **public** | **0.0%** | n/a | 9 | **9** |
| 23 | `CheckpointableRunBase` | internal (base of `Run`) | 42.8% | 16.6% | 14 | **8** |
| 24 | `Reflection.MessageHandlerInfo` | internal | 78.3% | 66.6% | 37 | **8** |
| 25 | `Specialized.ConcurrentEndExecutor` | internal | 78.9% | n/a | 38 | **8** |
### Other notable gaps
- **0% on small-but-public types**: `JsonCheckpointStore` (abstract base, 1
line, no test instantiates it), `IResettableExecutor` (interface default
method, 3 lines), `MagenticPlanReviewRequest`/`MagenticPlanReviewResponse`
(5/2 lines — partly covered by [#5833](https://github.com/microsoft/agent-framework/pull/5833)).
- **Obsolete attributes** with 0% coverage but no value to test:
`YieldsMessageAttribute`, `StreamsMessageAttribute`. Both are explicitly
marked `[Obsolete]` and are ignored by both the source generator and the
runtime — **excluded** from the plan.
- **Records / events** with low line counts (`SubworkflowWarningEvent`,
`RequestHaltEvent`, `ResetChatSignal`, `MagenticReplannedEvent`, etc.) appear
as 0% only because no test ever constructs them. Tiny absolute impact
individually, but easy bulk wins via constructor smoke tests.
- **`Observability.*`** classes appear at 043% because the entire
`ObservabilityTests` and `WorkflowRunActivityStopTests` files are skipped on
`main`. Re-enabling them via PR [#5837](https://github.com/microsoft/agent-framework/pull/5837)
is expected to lift those into the 8095% range without any new test code.
## Per-area coverage breakdown
(Aggregated by namespace; `Microsoft.Agents.AI.Workflows.Declarative*` excluded.)
| Area | Approx line cov | Notes |
| ---- | ---: | ----- |
| Top-level (`Workflow`, `WorkflowBuilder`, `WorkflowSession`, `WorkflowHostAgent`, `Run`, `StreamingRun`) | ~75% | Largest absolute gap; many public-surface gaps. |
| `Execution.*` | ~8590% on hot-path types, **<40%** on `EdgeConnection` and `ExecutorIdentity` | Identity/equality and connection-validation paths under-tested. |
| `Checkpointing.*` | ~80% mean | Several converter classes 1258%; `FileSystemJsonCheckpointStore` 62% (concurrency / corruption / index-rebuild paths). |
| `InProc.*` | ~85% | `InProcStepTracer` is the only outlier (62.9% line, 21.4% branch). |
| `Reflection.*` | ~85% | Solid; minor gaps. |
| `Visualization.*` | 92.1% | Already strong; remaining 18 lines are escape-handling and a couple of unusual edge layouts. |
| `Observability.*` | **<45%** today, expected **>85%** after [#5837](https://github.com/microsoft/agent-framework/pull/5837). |
| `Specialized.Magentic.*` | 3582% today; the orchestrator + builder + task context lift to high coverage after [#5833](https://github.com/microsoft/agent-framework/pull/5833). |
| `Specialized.*` (non-Magentic — handoff, group-chat, request-port helpers) | ~8090% | `RequestPortExtensions` (7.1%) is the only sharp outlier. |
| `Evaluation.*` | 89.4% | Mostly there; only branch coverage gaps remain. |
## Reproducing the per-class data
The full per-class table used to build the buckets above is regenerated by:
```bash
python3 - <<'PY'
import json
data = json.load(open('coverage-report/Summary.json'))
asm = data['coverage']['assemblies'][0]
def uncov(c): return c['coverablelines'] - c['coveredlines']
for c in sorted(asm['classesinassembly'], key=lambda c: -uncov(c)):
if uncov(c) == 0: continue
print(f"{c['name']:80s} {c['coverage']:5.1f}% uncov={uncov(c):4d} lines={c['coverablelines']}")
PY
```
@@ -9,42 +9,30 @@ namespace Harness.ConsoleReactiveComponents;
/// </summary>
public record TextPanelProps : ConsoleReactiveProps
{
/// <summary>Gets the items to render in the panel.</summary>
public IReadOnlyList<object> Items { get; init; } = [];
/// <summary>Gets the items to render in the panel. Each item is a pre-rendered
/// console string (may include ANSI escape sequences and newlines).</summary>
public IReadOnlyList<string> Items { get; init; } = [];
}
/// <summary>
/// A component that renders a list of items vertically using a custom render delegate.
/// A component that renders a list of pre-rendered string items vertically.
/// Designed for rendering dynamic items in a non-scroll region that may be
/// re-rendered on each update. If the component's <see cref="ConsoleReactiveComponent.Height"/>
/// exceeds the number of output lines, leftover lines are erased.
/// </summary>
public class TextPanel : ConsoleReactiveComponent<TextPanelProps, ConsoleReactiveState>
{
private readonly Func<object, string> _renderItem;
/// <summary>
/// Initializes a new instance of the <see cref="TextPanel"/> class.
/// </summary>
/// <param name="renderItem">A delegate that renders an item and returns the text to display (may contain newlines).</param>
public TextPanel(Func<object, string> renderItem)
{
this._renderItem = renderItem;
}
/// <summary>
/// Calculates the height (in lines) needed to render all items.
/// </summary>
/// <param name="items">The items to measure.</param>
/// <param name="renderItem">The render delegate to use for measuring.</param>
/// <returns>The total number of lines all items will occupy.</returns>
public static int CalculateHeight(IReadOnlyList<object> items, Func<object, string> renderItem)
public static int CalculateHeight(IReadOnlyList<string> items)
{
int total = 0;
for (int i = 0; i < items.Count; i++)
{
string text = renderItem(items[i]);
total += CountLines(text);
total += CountLines(items[i]);
}
return total;
@@ -57,7 +45,7 @@ public class TextPanel : ConsoleReactiveComponent<TextPanelProps, ConsoleReactiv
for (int i = 0; i < props.Items.Count; i++)
{
string text = this._renderItem(props.Items[i]);
string text = props.Items[i];
string[] lines = text.Split('\n');
int lineCount = CountLines(text);
@@ -9,8 +9,9 @@ namespace Harness.ConsoleReactiveComponents;
/// </summary>
public record TextScrollPanelProps : ConsoleReactiveProps
{
/// <summary>Gets the items to render in the scroll panel.</summary>
public IReadOnlyList<object> Items { get; init; } = [];
/// <summary>Gets the items to render in the scroll panel. Each item is a pre-rendered
/// console string (may include ANSI escape sequences and newlines).</summary>
public IReadOnlyList<string> Items { get; init; } = [];
}
/// <summary>
@@ -20,21 +21,17 @@ public record TextScrollPanelProps : ConsoleReactiveProps
public record TextScrollPanelState(int RenderedCount = 0) : ConsoleReactiveState;
/// <summary>
/// A component that renders items within a scroll area using a custom render delegate.
/// A component that renders pre-rendered string items within a scroll area.
/// All items are considered finalized — only new items since the last render are output.
/// Use <see cref="Reset"/> to force a full re-render.
/// </summary>
public class TextScrollPanel : ConsoleReactiveComponent<TextScrollPanelProps, TextScrollPanelState>
{
private readonly Func<object, string> _renderItem;
/// <summary>
/// Initializes a new instance of the <see cref="TextScrollPanel"/> class.
/// </summary>
/// <param name="renderItem">A delegate that renders a single item and returns the text to display (may contain newlines).</param>
public TextScrollPanel(Func<object, string> renderItem)
public TextScrollPanel()
{
this._renderItem = renderItem;
this.State = new TextScrollPanelState();
}
@@ -60,8 +57,7 @@ public class TextScrollPanel : ConsoleReactiveComponent<TextScrollPanelProps, Te
// Output only new items since last rendered
for (int i = state.RenderedCount; i < props.Items.Count; i++)
{
string text = this._renderItem(props.Items[i]);
Console.Write(text);
Console.Write(props.Items[i]);
}
// Update state to track what we've rendered
@@ -1,315 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using Harness.ConsoleReactiveComponents;
using Harness.ConsoleReactiveFramework;
namespace Harness.ConsoleSandbox;
/// <summary>
/// Determines which component is shown in the bottom panel.
/// </summary>
public enum BottomPanelMode
{
/// <summary>Show the list selection component.</summary>
ListSelection,
/// <summary>Show the text input component.</summary>
TextInput
}
public record AppComponentProps : ConsoleReactiveProps
{
public IReadOnlyList<string> Items { get; init; } = Array.Empty<string>();
public IReadOnlyList<object> ScrollItems { get; init; } = [];
/// <summary>Gets the bottom panel mode.</summary>
public BottomPanelMode Mode { get; init; } = BottomPanelMode.ListSelection;
/// <summary>Gets the prompt string for text input mode.</summary>
public string Prompt { get; init; } = "> ";
/// <summary>Gets the placeholder text shown when the input is empty.</summary>
public string Placeholder { get; init; } = "";
/// <summary>Gets the highlight color for the active list item. Defaults to <see cref="ConsoleColor.Cyan"/>.</summary>
public ConsoleColor ListHighlightColor { get; init; } = ConsoleColor.Cyan;
/// <summary>Gets the placeholder text for the custom text input option in the list. If <c>null</c>, no custom option is shown.</summary>
public string? ListCustomTextPlaceholder { get; init; }
/// <summary>Gets the foreground color for the rule borders. If <c>null</c>, uses the default terminal color.</summary>
public ConsoleColor? RuleColor { get; init; }
}
/// <summary>
/// Internal state for the <see cref="AppComponent"/>.
/// </summary>
public record AppComponentState : ConsoleReactiveState
{
/// <summary>Gets the selected index in list selection mode.</summary>
public int SelectedIndex { get; init; }
/// <summary>Gets the current input text being typed in text input mode.</summary>
public string InputText { get; init; } = "";
/// <summary>Gets the current text being typed into the list's custom text option.</summary>
public string ListInputText { get; init; } = "";
}
public class AppComponent : ConsoleReactiveComponent<AppComponentProps, AppComponentState>
{
private readonly TopBottomRule _rule = new();
private readonly ListSelection _listSelection = new();
private readonly TextInput _textInput = new();
private readonly TextScrollPanel _textScrollPanel;
private readonly TextPanel _textPanel;
private readonly Func<object, string> _renderItem;
private readonly Action<string> _onTextInputSubmit;
private readonly Action<string> _onListInputSubmit;
private bool _resizedSinceLastRender;
private int _lastScrollBottom;
/// <summary>
/// Initializes a new instance of the <see cref="AppComponent"/> class.
/// </summary>
/// <param name="renderScrollItem">A delegate that renders a single scroll panel item and returns the text to display.</param>
/// <param name="onTextInputSubmit">A callback invoked with the input text when the user presses Enter in text input mode.</param>
/// <param name="onListInputSubmit">A callback invoked with the selected or typed text when the user presses Enter in list selection mode.</param>
public AppComponent(Func<object, string> renderScrollItem, Action<string> onTextInputSubmit, Action<string> onListInputSubmit)
{
this._renderItem = renderScrollItem;
this._onTextInputSubmit = onTextInputSubmit;
this._onListInputSubmit = onListInputSubmit;
this._textScrollPanel = new TextScrollPanel(renderScrollItem);
this._textPanel = new TextPanel(renderScrollItem);
this.State = new AppComponentState();
KeyEventListener.Instance.KeyPressed += this.OnKeyPressed;
ConsoleResizeListener.Instance.ConsoleResized += this.OnConsoleResized;
}
private void OnKeyPressed(object? sender, KeyPressEventArgs e)
{
if (this.Props!.Mode == BottomPanelMode.TextInput)
{
this.HandleTextInputKey(e);
}
else
{
this.HandleListSelectionKey(e);
}
}
private void HandleTextInputKey(KeyPressEventArgs e)
{
if (e.KeyInfo.Key == ConsoleKey.Enter)
{
string text = this.State!.InputText;
this.SetState(this.State with { InputText = "" });
this._onTextInputSubmit(text);
}
else if (e.KeyInfo.Key == ConsoleKey.Backspace)
{
if (this.State!.InputText.Length > 0)
{
this.SetState(this.State with { InputText = this.State.InputText[..^1] });
}
}
else if (e.KeyInfo.KeyChar != '\0' && !char.IsControl(e.KeyInfo.KeyChar))
{
this.SetState(this.State! with { InputText = this.State.InputText + e.KeyInfo.KeyChar });
}
}
private void HandleListSelectionKey(KeyPressEventArgs e)
{
int maxIndex = this.Props!.Items.Count - 1;
if (this.Props.ListCustomTextPlaceholder != null)
{
maxIndex = this.Props.Items.Count; // extra option at the end
}
bool isOnCustomTextOption = this.Props.ListCustomTextPlaceholder != null
&& this.State!.SelectedIndex == this.Props.Items.Count;
if (e.KeyInfo.Key == ConsoleKey.UpArrow)
{
this.SetState(this.State! with { SelectedIndex = Math.Max(0, this.State.SelectedIndex - 1) });
}
else if (e.KeyInfo.Key == ConsoleKey.DownArrow)
{
this.SetState(this.State! with { SelectedIndex = Math.Min(maxIndex, this.State.SelectedIndex + 1) });
}
else if (e.KeyInfo.Key == ConsoleKey.Enter)
{
if (isOnCustomTextOption)
{
string text = this.State!.ListInputText;
this.SetState(this.State with { ListInputText = "" });
this._onListInputSubmit(text);
}
else
{
this._onListInputSubmit(this.Props.Items[this.State!.SelectedIndex]);
}
}
else if (isOnCustomTextOption)
{
// Typing only works when on the custom text option
if (e.KeyInfo.Key == ConsoleKey.Backspace)
{
if (this.State!.ListInputText.Length > 0)
{
this.SetState(this.State with { ListInputText = this.State.ListInputText[..^1] });
}
}
else if (e.KeyInfo.KeyChar != '\0' && !char.IsControl(e.KeyInfo.KeyChar))
{
this.SetState(this.State! with { ListInputText = this.State.ListInputText + e.KeyInfo.KeyChar });
}
}
}
private void OnConsoleResized(object? sender, ConsoleResizeEventArgs e)
{
this._resizedSinceLastRender = true;
this.Render();
}
public override void RenderCore(AppComponentProps props, AppComponentState state)
{
// Determine the text panel height for the last scroll item
object? lastItem = props.ScrollItems.Count > 0 ? props.ScrollItems[^1] : null;
IReadOnlyList<object> lastItems = lastItem != null ? [lastItem] : [];
int textPanelHeight = TextPanel.CalculateHeight(lastItems, this._renderItem);
if (textPanelHeight > 0)
{
textPanelHeight++; // Extra line for spacing between text panel and rule
}
// Build the bottom panel child based on mode
ConsoleReactiveComponent bottomChild;
int bottomChildHeight;
if (props.Mode == BottomPanelMode.TextInput)
{
var textInputProps = new TextInputProps
{
Prompt = props.Prompt,
Text = state.InputText,
Placeholder = props.Placeholder
};
bottomChildHeight = TextInput.CalculateHeight(textInputProps, Console.WindowWidth);
this._textInput.Width = Console.WindowWidth;
this._textInput.Height = bottomChildHeight;
this._textInput.Props = textInputProps;
bottomChild = this._textInput;
}
else
{
var listProps = new ListSelectionProps
{
Items = props.Items,
SelectedIndex = state.SelectedIndex,
HighlightColor = props.ListHighlightColor,
CustomTextPlaceholder = props.ListCustomTextPlaceholder,
CustomText = state.ListInputText
};
bottomChildHeight = ListSelection.CalculateHeight(listProps);
this._listSelection.Height = bottomChildHeight;
this._listSelection.Props = listProps;
bottomChild = this._listSelection;
}
var ruleProps = new TopBottomRuleProps
{
Width = Console.WindowWidth,
Color = props.RuleColor,
Children = [bottomChild]
};
int ruleHeight = TopBottomRule.CalculateHeight(ruleProps);
int scrollBottom = Console.WindowHeight - ruleHeight - textPanelHeight;
// If scroll region changed or a clear is needed, reset everything
if (this._resizedSinceLastRender || (this._lastScrollBottom != 0 && scrollBottom != this._lastScrollBottom))
{
Console.Write(AnsiEscapes.EraseEntireScreen);
Console.Write(AnsiEscapes.EraseScrollbackBuffer);
this._textScrollPanel.Reset();
this._resizedSinceLastRender = false;
}
this._lastScrollBottom = scrollBottom;
Console.Write(AnsiEscapes.SetScrollRegion(scrollBottom));
// Render text scroll panel in the scroll area (all items except the last)
IReadOnlyList<object> scrollItems = props.ScrollItems.Count > 1
? props.ScrollItems.Take(props.ScrollItems.Count - 1).ToList()
: [];
this._textScrollPanel.X = 1;
this._textScrollPanel.Y = 1;
this._textScrollPanel.Width = Console.WindowWidth;
this._textScrollPanel.Height = scrollBottom;
this._textScrollPanel.Props = new TextScrollPanelProps
{
Items = scrollItems
};
this._textScrollPanel.Render();
// Render the text panel for the last (dynamic) item just below the scroll region
this._textPanel.X = 1;
this._textPanel.Y = scrollBottom + 1;
this._textPanel.Width = Console.WindowWidth;
this._textPanel.Height = textPanelHeight;
this._textPanel.Props = new TextPanelProps
{
Items = lastItems,
};
this._textPanel.Render();
// Render the bottom rule + child below the text panel
this._rule.X = 1;
this._rule.Y = scrollBottom + textPanelHeight + 1;
this._rule.Props = ruleProps;
this._rule.Render();
// Position cursor for natural typing appearance
if (props.Mode == BottomPanelMode.TextInput)
{
int promptLength = props.Prompt.Length;
int textWidth = Console.WindowWidth - promptLength;
int textLength = state.InputText.Length;
// The TextInput starts at rule.Y + 1 (first row inside the rule)
int textInputY = this._rule.Y + 1;
if (textWidth <= 0 || textLength == 0)
{
// Cursor right after the prompt
Console.Write(AnsiEscapes.MoveCursor(textInputY, promptLength + 1));
}
else
{
// Calculate which row and column the cursor lands on
int cursorRow = textLength < textWidth ? 0 : 1 + ((textLength - textWidth) / textWidth);
int cursorCol = textLength < textWidth ? textLength : (textLength - textWidth) % textWidth;
Console.Write(AnsiEscapes.MoveCursor(textInputY + cursorRow, promptLength + cursorCol + 1));
}
}
else if (props.Mode == BottomPanelMode.ListSelection
&& props.ListCustomTextPlaceholder != null
&& state.SelectedIndex == props.Items.Count)
{
// Cursor after the typed text in the custom text option
// The custom text option is at rule.Y + 1 + Items.Count (0-based row inside rule)
int customOptionY = this._rule.Y + 1 + props.Items.Count;
// "> " prefix is 2 chars, then the typed text
int cursorCol = 2 + state.ListInputText.Length + 1;
Console.Write(AnsiEscapes.MoveCursor(customOptionY, cursorCol));
}
}
}
@@ -23,7 +23,7 @@ public abstract class CommandHandler
/// </summary>
/// <param name="input">The raw user input string.</param>
/// <param name="session">The current agent session.</param>
/// <param name="ux">The UX container for rendering output.</param>
/// <param name="ux">The UX state driver for rendering output.</param>
/// <returns><see langword="true"/> if this handler handled the input; <see langword="false"/> otherwise.</returns>
public abstract ValueTask<bool> TryHandleAsync(string input, AgentSession session, HarnessUXContainer ux);
public abstract ValueTask<bool> TryHandleAsync(string input, AgentSession session, IUXStateDriver ux);
}
@@ -0,0 +1,26 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI;
namespace Harness.Shared.Console.Commands;
/// <summary>
/// Handles the <c>/exit</c> command to shut down the console application.
/// </summary>
public sealed class ExitCommandHandler : CommandHandler
{
/// <inheritdoc/>
public override string? GetHelpText() => "/exit (quit)";
/// <inheritdoc/>
public override ValueTask<bool> TryHandleAsync(string input, AgentSession session, IUXStateDriver ux)
{
if (!input.Equals("/exit", StringComparison.OrdinalIgnoreCase))
{
return new ValueTask<bool>(false);
}
ux.RequestShutdown();
return new ValueTask<bool>(true);
}
}
@@ -7,7 +7,7 @@ namespace Harness.Shared.Console.Commands;
/// <summary>
/// Handles the <c>/mode</c> command to display or switch the current agent mode.
/// </summary>
internal sealed class ModeCommandHandler : CommandHandler
public sealed class ModeCommandHandler : CommandHandler
{
private readonly AgentModeProvider? _modeProvider;
private readonly IReadOnlyDictionary<string, ConsoleColor>? _modeColors;
@@ -27,7 +27,7 @@ internal sealed class ModeCommandHandler : CommandHandler
public override string? GetHelpText() => this._modeProvider is not null ? "/mode [plan|execute] (show or switch mode)" : null;
/// <inheritdoc/>
public override async ValueTask<bool> TryHandleAsync(string input, AgentSession session, HarnessUXContainer ux)
public override async ValueTask<bool> TryHandleAsync(string input, AgentSession session, IUXStateDriver ux)
{
if (!input.StartsWith("/mode ", StringComparison.OrdinalIgnoreCase) && !input.Equals("/mode", StringComparison.OrdinalIgnoreCase))
{
@@ -7,7 +7,7 @@ namespace Harness.Shared.Console.Commands;
/// <summary>
/// Handles the <c>/todos</c> command to display the current todo list.
/// </summary>
internal sealed class TodoCommandHandler : CommandHandler
public sealed class TodoCommandHandler : CommandHandler
{
private readonly TodoProvider? _todoProvider;
@@ -24,7 +24,7 @@ internal sealed class TodoCommandHandler : CommandHandler
public override string? GetHelpText() => this._todoProvider is not null ? "/todos (show todo list)" : null;
/// <inheritdoc/>
public override async ValueTask<bool> TryHandleAsync(string input, AgentSession session, HarnessUXContainer ux)
public override async ValueTask<bool> TryHandleAsync(string input, AgentSession session, IUXStateDriver ux)
{
if (!input.Equals("/todos", StringComparison.OrdinalIgnoreCase))
{
@@ -0,0 +1,59 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console;
/// <summary>
/// Represents an action returned by an observer at the end of an agent turn.
/// Subtypes describe either a question to ask the user (<see cref="FollowUpQuestion"/>)
/// or a message to add directly to the next agent input (<see cref="FollowUpMessage"/>).
/// </summary>
public abstract record FollowUpAction;
/// <summary>
/// Represents a question that should be presented to the user. The
/// <see cref="Continuation"/> delegate is invoked with the user's answer and the
/// UX state driver, and returns an optional <see cref="ChatMessage"/> to add to the
/// next agent invocation.
/// </summary>
/// <param name="Prompt">The question text shown to the user.</param>
/// <param name="Continuation">
/// Invoked with the user's answer and the UX state driver. The driver lets the
/// continuation write output (e.g., an action label like "Approved") in addition
/// to producing an optional <see cref="ChatMessage"/> for the next agent invocation.
/// </param>
public abstract record FollowUpQuestion(
string Prompt,
Func<string, IUXStateDriver, Task<ChatMessage?>> Continuation) : FollowUpAction;
/// <summary>
/// A free-form text question. The user may type any response.
/// </summary>
/// <param name="Prompt">The question text shown to the user.</param>
/// <param name="Continuation">Continuation that builds the response message.</param>
public sealed record TextFollowUpQuestion(
string Prompt,
Func<string, IUXStateDriver, Task<ChatMessage?>> Continuation)
: FollowUpQuestion(Prompt, Continuation);
/// <summary>
/// A choice question. The user picks from <paramref name="Choices"/>, optionally with
/// the ability to enter custom text when <paramref name="AllowCustomText"/> is true.
/// </summary>
/// <param name="Prompt">The question text shown to the user.</param>
/// <param name="Choices">The list of pre-defined choices.</param>
/// <param name="AllowCustomText">If true, the user may type a custom response in addition to the listed choices.</param>
/// <param name="Continuation">Continuation that builds the response message.</param>
public sealed record ChoiceFollowUpQuestion(
string Prompt,
IReadOnlyList<string> Choices,
bool AllowCustomText,
Func<string, IUXStateDriver, Task<ChatMessage?>> Continuation)
: FollowUpQuestion(Prompt, Continuation);
/// <summary>
/// A message to add directly to the next agent invocation without prompting the user.
/// </summary>
/// <param name="Message">The chat message to add.</param>
public sealed record FollowUpMessage(ChatMessage Message) : FollowUpAction;
@@ -0,0 +1,279 @@
// Copyright (c) Microsoft. All rights reserved.
using Harness.Shared.Console.Commands;
using Harness.Shared.Console.Observers;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console;
/// <summary>
/// Orchestrates agent invocations driven by user-input events from the UI.
/// The component invokes the runner's input handlers (<see cref="OnUserInputAsync"/>,
/// <see cref="OnStreamingInputAsync"/>, <see cref="StartAgentTurnAsync"/>) directly;
/// the runner mutates UI state through the supplied <see cref="IUXStateDriver"/>.
/// All per-turn follow-up state (pending questions and accumulated responses) lives
/// in the component's state record — the runner reads/writes it exclusively through
/// the driver and holds no per-turn fields itself.
/// </summary>
public sealed class HarnessAgentRunner : IDisposable
{
private readonly AIAgent _agent;
private readonly AgentSession _session;
private readonly AgentModeProvider? _modeProvider;
private readonly MessageInjectingChatClient? _messageInjector;
private readonly IReadOnlyList<CommandHandler> _commandHandlers;
private readonly IReadOnlyList<ConsoleObserver> _observers;
private readonly IUXStateDriver _ux;
private readonly SemaphoreSlim _inputGate = new(1, 1);
/// <summary>
/// Initializes a new instance of the <see cref="HarnessAgentRunner"/> class.
/// </summary>
public HarnessAgentRunner(
AIAgent agent,
AgentSession session,
AgentModeProvider? modeProvider,
MessageInjectingChatClient? messageInjector,
IReadOnlyList<CommandHandler> commandHandlers,
IReadOnlyList<ConsoleObserver> observers,
IUXStateDriver ux)
{
this._agent = agent;
this._session = session;
this._modeProvider = modeProvider;
this._messageInjector = messageInjector;
this._commandHandlers = commandHandlers;
this._observers = observers;
this._ux = ux;
this.HelpText = string.Join(
", ",
commandHandlers
.Select(h => h.GetHelpText())
.Where(t => t is not null)!);
}
/// <summary>
/// Gets the help text describing all available commands (joined by ", "), suitable
/// for display in the mode-and-help bar. Computed from the supplied
/// <c>commandHandlers</c>.
/// </summary>
public string HelpText { get; }
/// <inheritdoc/>
public void Dispose() => this._inputGate.Dispose();
/// <summary>
/// Handles a top-level user input submission (TextInput mode, no pending question).
/// Dispatches to command handlers, or starts an agent turn.
/// </summary>
internal async Task OnUserInputAsync(string text)
{
await this._inputGate.WaitAsync().ConfigureAwait(false);
try
{
this._ux.WriteUserInputEcho(text);
foreach (var handler in this._commandHandlers)
{
if (await handler.TryHandleAsync(text, this._session, this._ux).ConfigureAwait(false))
{
this._ux.CurrentMode = this._modeProvider?.GetMode(this._session);
return;
}
}
await this.RunAgentLoopAsync([new ChatMessage(ChatRole.User, text)]).ConfigureAwait(false);
}
finally
{
this._inputGate.Release();
}
}
/// <summary>
/// Handles a user input submission while an agent turn is streaming. The text is
/// enqueued via the <see cref="MessageInjectingChatClient"/> so it can be picked up
/// by the agent on its next opportunity.
/// </summary>
internal Task OnStreamingInputAsync(string text)
{
if (this._messageInjector is null)
{
return Task.CompletedTask;
}
this._messageInjector.EnqueueMessages(this._session, [new ChatMessage(ChatRole.User, text)]);
this._ux.SetQueuedMessages(this._messageInjector.GetPendingMessages(this._session));
return Task.CompletedTask;
}
/// <summary>
/// Resumes (or completes) a turn after the user has answered all pending follow-up
/// questions. The component invokes this with the messages drained from
/// <see cref="IUXStateDriver.TakeFollowUpResponses"/>; an empty list simply ends
/// the streaming display state without invoking the agent.
/// </summary>
internal async Task StartAgentTurnAsync(IList<ChatMessage> messages)
{
await this._inputGate.WaitAsync().ConfigureAwait(false);
try
{
if (messages.Count == 0)
{
this.CompleteTurn();
return;
}
await this.RunAgentLoopAsync(messages).ConfigureAwait(false);
}
finally
{
this._inputGate.Release();
}
}
private async Task RunAgentLoopAsync(IList<ChatMessage> messages)
{
IList<ChatMessage>? nextMessages = messages;
IReadOnlyList<ChatMessage> lastPendingMessages = this._messageInjector?.GetPendingMessages(this._session) ?? [];
while (nextMessages is not null)
{
var runOptions = new AgentRunOptions();
foreach (var observer in this._observers)
{
observer.ConfigureRunOptions(runOptions, this._agent, this._session);
}
this._ux.CurrentMode = this._modeProvider?.GetMode(this._session);
this._ux.BeginStreaming();
this._ux.BeginStreamingOutput();
try
{
await foreach (var update in this._agent.RunStreamingAsync(nextMessages, this._session, runOptions))
{
if (this._modeProvider is not null)
{
string currentMode = this._modeProvider.GetMode(this._session);
if (currentMode != this._ux.CurrentMode)
{
this._ux.CurrentMode = currentMode;
}
}
foreach (var content in update.Contents)
{
foreach (var observer in this._observers)
{
await observer.OnContentAsync(this._ux, content, this._agent, this._session).ConfigureAwait(false);
}
}
if (!string.IsNullOrEmpty(update.Text))
{
foreach (var observer in this._observers)
{
await observer.OnTextAsync(this._ux, update.Text, this._agent, this._session).ConfigureAwait(false);
}
}
this.SyncQueuedMessageDisplay(ref lastPendingMessages);
}
}
catch (Exception ex)
{
await this._ux.WriteInfoLineAsync($"❌ Stream error: {ex.GetType().Name}:\n{ex}", ConsoleColor.Red).ConfigureAwait(false);
}
// Final sync after streaming.
this.SyncQueuedMessageDisplay(ref lastPendingMessages);
this._ux.StopSpinner();
await this._ux.EndStreamingOutputAsync().ConfigureAwait(false);
// Collect FollowUpActions from each observer.
var directMessages = new List<ChatMessage>();
var questions = new List<FollowUpQuestion>();
foreach (var observer in this._observers)
{
var actions = await observer.OnStreamCompleteAsync(this._ux, this._agent, this._session).ConfigureAwait(false);
if (actions is null)
{
continue;
}
foreach (var action in actions)
{
switch (action)
{
case FollowUpMessage msg:
directMessages.Add(msg.Message);
break;
case FollowUpQuestion q:
questions.Add(q);
break;
}
}
}
bool hasFollowUpActions = directMessages.Count > 0 || questions.Count > 0;
await this._ux.WriteNoTextWarningAsync(hasFollowUpActions).ConfigureAwait(false);
// Add any direct messages to the accumulator regardless of whether questions follow —
// they're sent on the next agent invocation, either by us (if no questions) or by
// the component (after the user finishes answering, via StartAgentTurnAsync).
foreach (var msg in directMessages)
{
this._ux.AddFollowUpResponse(msg);
}
if (questions.Count > 0)
{
// Pause: hand control back to the UX to collect answers.
this._ux.QueueFollowUpQuestions(questions);
return;
}
// No questions to ask — drain anything we just accumulated and loop with it.
IReadOnlyList<ChatMessage> drained = this._ux.TakeFollowUpResponses();
nextMessages = drained.Count > 0 ? [.. drained] : null;
}
this.CompleteTurn();
}
private void CompleteTurn()
{
this._ux.EndStreaming();
this._ux.CurrentMode = this._modeProvider?.GetMode(this._session);
}
/// <summary>
/// Synchronizes the queued items display with the message injector's pending messages.
/// Messages that have been consumed (drained by the service) are echoed to the output
/// area as regular user-input entries.
/// </summary>
private void SyncQueuedMessageDisplay(ref IReadOnlyList<ChatMessage> lastPendingMessages)
{
if (this._messageInjector is null)
{
return;
}
var pending = this._messageInjector.GetPendingMessages(this._session);
int consumedCount = lastPendingMessages.Count - pending.Count;
for (int i = 0; i < consumedCount && i < lastPendingMessages.Count; i++)
{
string text = lastPendingMessages[i].Text ?? string.Empty;
this._ux.WriteUserInputEcho(text);
}
lastPendingMessages = pending;
this._ux.SetQueuedMessages(pending);
}
}
@@ -3,171 +3,89 @@
using Harness.ConsoleReactiveComponents;
using Harness.ConsoleReactiveFramework;
using Harness.Shared.Console.Components;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console;
/// <summary>
/// Determines which component is shown in the bottom panel.
/// </summary>
public enum BottomPanelMode
{
/// <summary>Show the text input component for user input.</summary>
TextInput,
/// <summary>Show the list selection component for interactive prompts.</summary>
ListSelection,
/// <summary>Show a disabled input indicator during agent streaming.</summary>
Streaming,
}
/// <summary>
/// Event arguments for the <see cref="HarnessAppComponent.InputSubmitted"/> event.
/// </summary>
public sealed class InputSubmittedEventArgs : EventArgs
{
/// <summary>
/// Initializes a new instance of the <see cref="InputSubmittedEventArgs"/> class.
/// </summary>
/// <param name="text">The submitted text.</param>
/// <param name="mode">The bottom panel mode in which the input was submitted.</param>
public InputSubmittedEventArgs(string text, BottomPanelMode mode)
{
this.Text = text;
this.Mode = mode;
}
/// <summary>Gets the submitted text.</summary>
public string Text { get; }
/// <summary>Gets the bottom panel mode in which the input was submitted.</summary>
public BottomPanelMode Mode { get; }
}
/// <summary>
/// Props for <see cref="HarnessAppComponent"/>.
/// </summary>
public record HarnessAppComponentProps : ConsoleReactiveProps
{
/// <summary>Gets or sets the list selection choices (for ListSelection mode).</summary>
public IReadOnlyList<string> Items { get; set; } = Array.Empty<string>();
/// <summary>Gets or sets the scroll items (output entries) to render in the scroll panel.</summary>
public IReadOnlyList<object> ScrollItems { get; set; } = [];
/// <summary>Gets or sets the bottom panel mode.</summary>
public BottomPanelMode Mode { get; set; } = BottomPanelMode.TextInput;
/// <summary>Gets or sets the prompt string for text input mode.</summary>
public string Prompt { get; set; } = "You: ";
/// <summary>Gets or sets the placeholder text shown when the input is empty.</summary>
public string Placeholder { get; set; } = "";
/// <summary>Gets or sets the highlight color for the active list item.</summary>
public ConsoleColor ListHighlightColor { get; set; } = ConsoleColor.Cyan;
/// <summary>Gets or sets the placeholder text for the custom text input option in the list.</summary>
public string? ListCustomTextPlaceholder { get; set; }
/// <summary>Gets or sets the foreground color for the rule borders and mode label.</summary>
public ConsoleColor? ModeColor { get; set; }
/// <summary>Gets or sets the current mode name displayed below the bottom rule (e.g. "plan").</summary>
public string? ModeText { get; set; }
/// <summary>Gets or sets the help text displayed below the bottom rule (available commands).</summary>
public string? HelpText { get; set; }
/// <summary>Gets or sets the title text displayed above the list selection (for interactive prompts).</summary>
public string? ListTitle { get; set; }
/// <summary>Gets or sets a value indicating whether input is enabled during streaming.</summary>
public bool InputEnabled { get; set; }
/// <summary>Gets or sets the prompt to show during streaming when input is disabled.</summary>
public string StreamingPrompt { get; set; } = "(agent is running...)";
/// <summary>Gets or sets a value indicating whether the agent status spinner is visible.</summary>
public bool ShowSpinner { get; set; }
/// <summary>Gets or sets the formatted token usage text to display in the status bar.</summary>
public string? UsageText { get; set; }
/// <summary>Gets or sets the queued input items to display above the rule.</summary>
public IReadOnlyList<object> QueuedItems { get; set; } = [];
}
/// <summary>
/// Internal state for <see cref="HarnessAppComponent"/>.
/// </summary>
public record HarnessAppComponentState : ConsoleReactiveState
{
/// <summary>Gets the selected index in list selection mode.</summary>
public int SelectedIndex { get; init; }
/// <summary>Gets the current input text being typed.</summary>
public string InputText { get; init; } = "";
/// <summary>Gets the current text being typed into the list's custom text option.</summary>
public string ListInputText { get; init; } = "";
/// <summary>Gets the current console width in columns.</summary>
public int ConsoleWidth { get; init; }
/// <summary>Gets the current console height in rows.</summary>
public int ConsoleHeight { get; init; }
}
/// <summary>
/// The main application component for the Harness console. Manages the scroll region
/// and bottom panel (text input, list selection, or streaming indicator), and emits
/// an <see cref="InputSubmitted"/> event when the user submits text in any mode.
/// and bottom panel (text input, list selection, or streaming indicator). Owns the
/// <see cref="HarnessConsoleUXStateDriver"/> and routes user input events to the
/// registered <see cref="HarnessAgentRunner"/>.
/// </summary>
public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentProps, HarnessAppComponentState>, IDisposable
public class HarnessAppComponent : ConsoleReactiveComponent<ConsoleReactiveProps, HarnessAppComponentState>, IDisposable
{
private readonly TopBottomRule _rule = new();
private readonly ListSelection _listSelection = new();
private readonly TextInput _textInput = new();
private readonly TextScrollPanel _textScrollPanel;
private readonly TextPanel _textPanel;
private readonly TextPanel _queuedPanel;
private readonly TextScrollPanel _textScrollPanel = new();
private readonly TextPanel _textPanel = new();
private readonly TextPanel _queuedPanel = new();
private readonly AgentStatus _agentStatus = new();
private readonly AgentModeAndHelp _modeAndHelp = new();
private readonly Func<object, string> _renderItem;
private bool _resizedSinceLastRender;
private readonly HarnessConsoleUXStateDriver _uxDriver;
private readonly TaskCompletionSource<bool> _shutdownTcs = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly SemaphoreSlim _followUpGate = new(1, 1);
private int _scrollRegionBottom;
private bool _resizedSinceLastRender = true;
private bool _deactivated;
/// <summary>
/// Initializes a new instance of the <see cref="HarnessAppComponent"/> class.
/// </summary>
/// <param name="renderScrollItem">A delegate that renders a single output entry and returns the text to display.</param>
public HarnessAppComponent(Func<object, string> renderScrollItem)
/// <param name="placeholder">Placeholder text shown when the input is empty.</param>
/// <param name="initialMode">The current agent mode, used to colour the rule and prompt.</param>
/// <param name="inputEnabled">Whether the bottom-panel input accepts keystrokes during streaming.</param>
/// <param name="runnerFactory">Factory invoked with the component's <see cref="IUXStateDriver"/>
/// to construct the <see cref="HarnessAgentRunner"/> that owns the agent loop.</param>
/// <param name="modeColors">Optional mapping of mode names to console colors.</param>
public HarnessAppComponent(
string placeholder,
string? initialMode,
bool inputEnabled,
Func<IUXStateDriver, HarnessAgentRunner> runnerFactory,
IReadOnlyDictionary<string, ConsoleColor>? modeColors = null)
{
this._renderItem = renderScrollItem;
this._textScrollPanel = new TextScrollPanel(renderScrollItem);
this._textPanel = new TextPanel(renderScrollItem);
this._queuedPanel = new TextPanel(renderScrollItem);
this.Props = new ConsoleReactiveProps();
this.State = new HarnessAppComponentState
{
Mode = BottomPanelMode.TextInput,
Prompt = "> ",
Placeholder = placeholder,
ModeColor = ModeColors.Get(initialMode, modeColors),
ModeText = initialMode,
InputEnabled = inputEnabled,
ConsoleWidth = System.Console.WindowWidth,
ConsoleHeight = System.Console.WindowHeight,
};
this._uxDriver = new HarnessConsoleUXStateDriver(
getState: () => this.State!,
setState: s => this.SetState(s),
requestShutdown: () => this._shutdownTcs.TrySetResult(true),
modeColors: modeColors);
this.Runner = runnerFactory(this._uxDriver);
// Seed help text now that the runner (which knows the registered command handlers)
// is available. Direct assignment — no Render is triggered until the caller invokes Render().
this.State = this.State with { HelpText = this.Runner.HelpText };
KeyEventListener.Instance.KeyPressed += this.OnKeyPressed;
ConsoleResizeListener.Instance.ConsoleResized += this.OnConsoleResized;
}
/// <summary>
/// Gets the 1-based row number of the last row in the output scroll region.
/// Gets the agent runner that owns the agent loop. Constructed by the factory
/// passed to the component's constructor.
/// </summary>
public int ScrollRegionBottom { get; private set; }
public HarnessAgentRunner Runner { get; }
/// <summary>
/// Occurs when the user submits input via Enter, in any mode (text input, list selection,
/// or streaming injection). Consumers inspect <see cref="InputSubmittedEventArgs.Mode"/>
/// to decide how to handle the submission.
/// Completes when a command handler requests application shutdown (e.g. the user types <c>/exit</c>).
/// Awaited by <see cref="HarnessConsole.RunAgentAsync"/>.
/// </summary>
public event EventHandler<InputSubmittedEventArgs>? InputSubmitted;
public Task ShutdownTask => this._shutdownTcs.Task;
/// <summary>
/// Deactivates the component, resetting the scroll region and unsubscribing from events.
@@ -184,9 +102,6 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
this._agentStatus.Dispose();
KeyEventListener.Instance.KeyPressed -= this.OnKeyPressed;
ConsoleResizeListener.Instance.ConsoleResized -= this.OnConsoleResized;
System.Console.Write(AnsiEscapes.ResetScrollRegion);
System.Console.Write(AnsiEscapes.MoveCursor(System.Console.WindowHeight, 1));
System.Console.WriteLine();
}
/// <inheritdoc/>
@@ -205,20 +120,23 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
if (disposing)
{
this.Deactivate();
this._followUpGate.Dispose();
this.Runner.Dispose();
}
}
private void OnKeyPressed(object? sender, KeyPressEventArgs e)
{
if (this.Props!.Mode == BottomPanelMode.TextInput)
BottomPanelMode mode = this.State!.Mode;
if (mode == BottomPanelMode.TextInput)
{
this.HandleTextInputKey(e);
}
else if (this.Props.Mode == BottomPanelMode.ListSelection)
else if (mode == BottomPanelMode.ListSelection)
{
this.HandleListSelectionKey(e);
}
else if (this.Props.Mode == BottomPanelMode.Streaming && this.Props.InputEnabled)
else if (mode == BottomPanelMode.Streaming && this.State.InputEnabled)
{
this.HandleStreamingInputKey(e);
}
@@ -235,7 +153,7 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
}
this.SetState(this.State with { InputText = "" });
this.InputSubmitted?.Invoke(this, new InputSubmittedEventArgs(text, BottomPanelMode.TextInput));
this.DispatchTextInputSubmission(text);
}
else if (e.KeyInfo.Key == ConsoleKey.Backspace)
{
@@ -252,51 +170,50 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
private void HandleListSelectionKey(KeyPressEventArgs e)
{
int maxIndex = this.Props!.Items.Count - 1;
if (this.Props.ListCustomTextPlaceholder != null)
int maxIndex = this.State!.ListSelectionOptions.Count - 1;
if (this.State.ListSelectionCustomTextPlaceholder != null)
{
maxIndex = this.Props.Items.Count;
maxIndex = this.State.ListSelectionOptions.Count;
}
bool isOnCustomTextOption = this.Props.ListCustomTextPlaceholder != null
&& this.State!.SelectedIndex == this.Props.Items.Count;
bool isOnCustomTextOption = this.State.ListSelectionCustomTextPlaceholder != null
&& this.State.ListSelectionIndex == this.State.ListSelectionOptions.Count;
if (e.KeyInfo.Key == ConsoleKey.UpArrow)
{
this.SetState(this.State! with { SelectedIndex = Math.Max(0, this.State.SelectedIndex - 1) });
this.SetState(this.State with { ListSelectionIndex = Math.Max(0, this.State.ListSelectionIndex - 1) });
}
else if (e.KeyInfo.Key == ConsoleKey.DownArrow)
{
this.SetState(this.State! with { SelectedIndex = Math.Min(maxIndex, this.State.SelectedIndex + 1) });
this.SetState(this.State with { ListSelectionIndex = Math.Min(maxIndex, this.State.ListSelectionIndex + 1) });
}
else if (e.KeyInfo.Key == ConsoleKey.Enter)
{
string result = isOnCustomTextOption
? this.State!.ListInputText
: this.Props.Items[this.State!.SelectedIndex];
? this.State.ListSelectionCustomInputText
: this.State.ListSelectionOptions[this.State.ListSelectionIndex];
this.SetState(this.State with { ListInputText = "", SelectedIndex = 0 });
this.InputSubmitted?.Invoke(this, new InputSubmittedEventArgs(result, BottomPanelMode.ListSelection));
this.SetState(this.State with { ListSelectionCustomInputText = "", ListSelectionIndex = 0 });
this.DispatchListSelectionSubmission(result);
}
else if (isOnCustomTextOption)
{
if (e.KeyInfo.Key == ConsoleKey.Backspace)
{
if (this.State!.ListInputText.Length > 0)
if (this.State.ListSelectionCustomInputText.Length > 0)
{
this.SetState(this.State with { ListInputText = this.State.ListInputText[..^1] });
this.SetState(this.State with { ListSelectionCustomInputText = this.State.ListSelectionCustomInputText[..^1] });
}
}
else if (e.KeyInfo.KeyChar != '\0' && !char.IsControl(e.KeyInfo.KeyChar))
{
this.SetState(this.State! with { ListInputText = this.State.ListInputText + e.KeyInfo.KeyChar });
this.SetState(this.State with { ListSelectionCustomInputText = this.State.ListSelectionCustomInputText + e.KeyInfo.KeyChar });
}
}
}
private void HandleStreamingInputKey(KeyPressEventArgs e)
{
// During streaming with input enabled, capture text for message injection
if (e.KeyInfo.Key == ConsoleKey.Enter)
{
string text = this.State!.InputText;
@@ -306,7 +223,7 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
}
this.SetState(this.State with { InputText = "" });
this.InputSubmitted?.Invoke(this, new InputSubmittedEventArgs(text, BottomPanelMode.Streaming));
_ = this.Runner.OnStreamingInputAsync(text);
}
else if (e.KeyInfo.Key == ConsoleKey.Backspace)
{
@@ -321,6 +238,90 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
}
}
private void DispatchTextInputSubmission(string text)
{
if (this.State!.PendingQuestions.Count > 0)
{
_ = this.HandleFollowUpAnswerAsync(text);
}
else
{
_ = this.Runner.OnUserInputAsync(text);
}
}
private void DispatchListSelectionSubmission(string text)
{
// List selection is only used to answer FollowUpQuestions.
_ = this.HandleFollowUpAnswerAsync(text);
}
/// <summary>
/// Handles a user answer to the head of the pending follow-up question queue:
/// awaits the question's continuation (which is responsible for echoing both the
/// question and answer to the scroll area as it sees fit), appends any returned
/// chat message to the response accumulator, advances the queue, and — when the
/// queue empties — drains the accumulator and resumes the runner.
/// </summary>
private async Task HandleFollowUpAnswerAsync(string text)
{
IReadOnlyList<ChatMessage>? messagesToSend = null;
await this._followUpGate.WaitAsync().ConfigureAwait(false);
try
{
HarnessConsoleUXStateDriver ux = this._uxDriver;
IReadOnlyList<FollowUpQuestion> queue = this.State!.PendingQuestions;
if (queue.Count == 0)
{
return;
}
FollowUpQuestion head = queue[0];
ChatMessage? response;
try
{
response = await head.Continuation(text, ux).ConfigureAwait(false);
}
catch (Exception ex)
{
await ux.WriteInfoLineAsync($"❌ Follow-up handler error: {ex.GetType().Name}: {ex.Message}", ConsoleColor.Red).ConfigureAwait(false);
response = null;
}
if (response is not null)
{
ux.AddFollowUpResponse(response);
}
ux.AdvanceFollowUpQuestion();
if (this.State!.PendingQuestions.Count == 0)
{
messagesToSend = ux.TakeFollowUpResponses();
}
}
finally
{
this._followUpGate.Release();
}
// Resume the agent outside the gate — StartAgentTurnAsync runs the full agent
// loop which may queue new follow-up questions (re-entering this method).
if (messagesToSend is not null)
{
try
{
await this.Runner.StartAgentTurnAsync([.. messagesToSend]).ConfigureAwait(false);
}
catch (Exception ex)
{
await this._uxDriver.WriteInfoLineAsync($"❌ Agent error: {ex.GetType().Name}: {ex.Message}", ConsoleColor.Red).ConfigureAwait(false);
}
}
}
private void OnConsoleResized(object? sender, ConsoleResizeEventArgs e)
{
this._resizedSinceLastRender = true;
@@ -332,35 +333,40 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
}
/// <inheritdoc />
public override void RenderCore(HarnessAppComponentProps props, HarnessAppComponentState state)
public override void RenderCore(ConsoleReactiveProps props, HarnessAppComponentState state)
{
if (this._deactivated)
{
return;
}
// Determine the text panel height for the last scroll item
IReadOnlyList<object> lastItems = props.ScrollItems.Count > 0
? [props.ScrollItems[^1]]
IReadOnlyList<string> lastItems = state.ScrollAreaContentItems.Count > 0
? [state.ScrollAreaContentItems[^1]]
: [];
int textPanelHeight = TextPanel.CalculateHeight(lastItems, this._renderItem);
int textPanelHeight = TextPanel.CalculateHeight(lastItems);
if (textPanelHeight > 0)
{
textPanelHeight++; // Extra line for spacing between text panel and rule
}
// Calculate queued items panel height
int queuedPanelHeight = TextPanel.CalculateHeight(props.QueuedItems, this._renderItem);
int queuedPanelHeight = TextPanel.CalculateHeight(state.QueuedItems);
// Build the bottom panel child based on mode
ConsoleReactiveComponent bottomChild;
int bottomChildHeight;
if (props.Mode == BottomPanelMode.ListSelection)
if (state.Mode == BottomPanelMode.ListSelection)
{
var listProps = new ListSelectionProps
{
Title = props.ListTitle,
Items = props.Items,
SelectedIndex = state.SelectedIndex,
HighlightColor = props.ListHighlightColor,
CustomTextPlaceholder = props.ListCustomTextPlaceholder,
CustomText = state.ListInputText,
Title = state.ListSelectionTitle,
Items = state.ListSelectionOptions,
SelectedIndex = state.ListSelectionIndex,
HighlightColor = state.ListHighlightColor,
CustomTextPlaceholder = state.ListSelectionCustomTextPlaceholder,
CustomText = state.ListSelectionCustomInputText,
};
bottomChildHeight = ListSelection.CalculateHeight(listProps);
@@ -368,25 +374,25 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
this._listSelection.Props = listProps;
bottomChild = this._listSelection;
}
else if (props.Mode == BottomPanelMode.Streaming)
else if (state.Mode == BottomPanelMode.Streaming)
{
TextInputProps textInputProps;
if (props.InputEnabled)
if (state.InputEnabled)
{
textInputProps = new TextInputProps
{
Prompt = props.Prompt,
Prompt = state.Prompt,
Text = state.InputText,
Placeholder = props.Placeholder,
Placeholder = state.Placeholder,
};
}
else
{
textInputProps = new TextInputProps
{
Prompt = props.Prompt,
Prompt = state.Prompt,
Text = "",
Placeholder = props.StreamingPrompt,
Placeholder = state.StreamingPrompt,
};
}
@@ -400,9 +406,9 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
{
var textInputProps = new TextInputProps
{
Prompt = props.Prompt,
Prompt = state.Prompt,
Text = state.InputText,
Placeholder = props.Placeholder,
Placeholder = state.Placeholder,
};
bottomChildHeight = TextInput.CalculateHeight(textInputProps, state.ConsoleWidth);
@@ -415,46 +421,52 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
var ruleProps = new TopBottomRuleProps
{
Width = state.ConsoleWidth,
Color = props.ModeColor,
Color = state.ModeColor,
Children = [bottomChild],
};
// Calculate the agent status height
var agentStatusProps = new AgentStatusProps
{
ShowSpinner = props.ShowSpinner,
UsageText = props.UsageText,
ShowSpinner = state.ShowSpinner,
UsageText = state.UsageText,
};
int agentStatusHeight = AgentStatus.CalculateHeight(agentStatusProps);
// Calculate the mode-and-help height
var modeAndHelpProps = new AgentModeAndHelpProps
{
Mode = props.ModeText,
ModeColor = props.ModeColor,
HelpText = props.HelpText,
Mode = state.ModeText,
ModeColor = state.ModeColor,
HelpText = state.HelpText,
};
int modeAndHelpHeight = AgentModeAndHelp.CalculateHeight(modeAndHelpProps);
// Hide agent status and mode/help during follow-up questions (ListSelection mode)
// as they clutter the UI and aren't relevant.
bool showStatusAndHelp = state.Mode != BottomPanelMode.ListSelection;
int agentStatusHeight = showStatusAndHelp ? AgentStatus.CalculateHeight(agentStatusProps) : 0;
int modeAndHelpHeight = showStatusAndHelp ? AgentModeAndHelp.CalculateHeight(modeAndHelpProps) : 0;
int ruleHeight = TopBottomRule.CalculateHeight(ruleProps);
int scrollBottom = Math.Max(1, state.ConsoleHeight - ruleHeight - textPanelHeight - agentStatusHeight - queuedPanelHeight - modeAndHelpHeight);
int nonScrollHeight = ruleHeight + textPanelHeight + agentStatusHeight + queuedPanelHeight + modeAndHelpHeight + 1; // +1 for bottom padding
int scrollBottom = Math.Max(1, state.ConsoleHeight - nonScrollHeight);
// If scroll region changed or a clear is needed, reset everything
if (this._resizedSinceLastRender || (this.ScrollRegionBottom != 0 && scrollBottom != this.ScrollRegionBottom))
if (this._resizedSinceLastRender || (this._scrollRegionBottom != 0 && scrollBottom != this._scrollRegionBottom))
{
// Reset scroll region to full screen before erasing so the erase covers all rows —
// some terminals only erase within the active DECSTBM region.
System.Console.Write(AnsiEscapes.ResetScrollRegion);
System.Console.Write(AnsiEscapes.EraseEntireScreen);
System.Console.Write(AnsiEscapes.EraseScrollbackBuffer);
this._textScrollPanel.Reset();
this._resizedSinceLastRender = false;
}
this.ScrollRegionBottom = scrollBottom;
this._scrollRegionBottom = scrollBottom;
System.Console.Write(AnsiEscapes.SetScrollRegion(scrollBottom));
// Render text scroll panel in the scroll area (all items except the last)
IReadOnlyList<object> scrollItems = props.ScrollItems.Count > 1
? props.ScrollItems.Take(props.ScrollItems.Count - 1).ToList()
IReadOnlyList<string> scrollItems = state.ScrollAreaContentItems.Count > 1
? state.ScrollAreaContentItems.Take(state.ScrollAreaContentItems.Count - 1).ToList()
: [];
this._textScrollPanel.X = 1;
@@ -486,18 +498,21 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
this._queuedPanel.Height = queuedPanelHeight;
this._queuedPanel.Props = new TextPanelProps
{
Items = props.QueuedItems,
Items = state.QueuedItems,
};
this._queuedPanel.Render();
// Render the agent status line between queued items and rule
int agentStatusY = queuedPanelY + queuedPanelHeight;
this._agentStatus.X = 1;
this._agentStatus.Y = agentStatusY;
this._agentStatus.Width = state.ConsoleWidth;
this._agentStatus.Height = agentStatusHeight;
this._agentStatus.Props = agentStatusProps;
this._agentStatus.Render();
if (showStatusAndHelp)
{
this._agentStatus.X = 1;
this._agentStatus.Y = agentStatusY;
this._agentStatus.Width = state.ConsoleWidth;
this._agentStatus.Height = agentStatusHeight;
this._agentStatus.Props = agentStatusProps;
this._agentStatus.Render();
}
// Render the bottom rule + child below the agent status
this._rule.X = 1;
@@ -506,24 +521,27 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
this._rule.Render();
// Render the mode-and-help line below the bottom rule
int modeAndHelpY = this._rule.Y + ruleHeight;
this._modeAndHelp.X = 1;
this._modeAndHelp.Y = modeAndHelpY;
this._modeAndHelp.Width = state.ConsoleWidth;
this._modeAndHelp.Height = modeAndHelpHeight;
this._modeAndHelp.Props = modeAndHelpProps;
this._modeAndHelp.Render();
if (showStatusAndHelp)
{
int modeAndHelpY = this._rule.Y + ruleHeight;
this._modeAndHelp.X = 1;
this._modeAndHelp.Y = modeAndHelpY;
this._modeAndHelp.Width = state.ConsoleWidth;
this._modeAndHelp.Height = modeAndHelpHeight;
this._modeAndHelp.Props = modeAndHelpProps;
this._modeAndHelp.Render();
}
// Position cursor for natural typing appearance
this.PositionCursor(props, state);
this.PositionCursor(state);
}
private void PositionCursor(HarnessAppComponentProps props, HarnessAppComponentState state)
private void PositionCursor(HarnessAppComponentState state)
{
if (props.Mode == BottomPanelMode.TextInput
|| (props.Mode == BottomPanelMode.Streaming && props.InputEnabled))
if (state.Mode == BottomPanelMode.TextInput
|| (state.Mode == BottomPanelMode.Streaming && state.InputEnabled))
{
int promptLength = props.Prompt.Length;
int promptLength = state.Prompt.Length;
int textWidth = state.ConsoleWidth - promptLength;
int textLength = state.InputText.Length;
@@ -540,13 +558,13 @@ public class HarnessAppComponent : ConsoleReactiveComponent<HarnessAppComponentP
System.Console.Write(AnsiEscapes.MoveCursor(textInputY + cursorRow, promptLength + cursorCol + 1));
}
}
else if (props.Mode == BottomPanelMode.ListSelection
&& props.ListCustomTextPlaceholder != null
&& state.SelectedIndex == props.Items.Count)
else if (state.Mode == BottomPanelMode.ListSelection
&& state.ListSelectionCustomTextPlaceholder != null
&& state.ListSelectionIndex == state.ListSelectionOptions.Count)
{
int titleLines = props.ListTitle?.Split('\n').Length ?? 0;
int customOptionY = this._rule.Y + 1 + titleLines + props.Items.Count;
int cursorCol = 2 + state.ListInputText.Length + 1;
int titleLines = state.ListSelectionTitle?.Split('\n').Length ?? 0;
int customOptionY = this._rule.Y + 1 + titleLines + state.ListSelectionOptions.Count;
int cursorCol = 2 + state.ListSelectionCustomInputText.Length + 1;
System.Console.Write(AnsiEscapes.MoveCursor(customOptionY, cursorCol));
}
}
@@ -0,0 +1,125 @@
// Copyright (c) Microsoft. All rights reserved.
using Harness.ConsoleReactiveFramework;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console;
/// <summary>
/// Determines which component is shown in the bottom panel.
/// </summary>
public enum BottomPanelMode
{
/// <summary>Show the text input component for user input.</summary>
TextInput,
/// <summary>Show the list selection component for interactive prompts.</summary>
ListSelection,
/// <summary>Show a disabled input indicator during agent streaming.</summary>
Streaming,
}
/// <summary>
/// Internal state for <see cref="HarnessAppComponent"/>. All UI fields that may
/// change after construction live here; they are mutated exclusively via
/// <see cref="ConsoleReactiveComponent{TProps,TState}.SetState"/> by the
/// owning <see cref="HarnessConsoleUXStateDriver"/>.
/// </summary>
public record HarnessAppComponentState : ConsoleReactiveState
{
// --- Console dimensions ---
/// <summary>Gets the current console width in columns.</summary>
public int ConsoleWidth { get; init; }
/// <summary>Gets the current console height in rows.</summary>
public int ConsoleHeight { get; init; }
// --- Bottom panel mode ---
/// <summary>Gets the bottom panel mode.</summary>
public BottomPanelMode Mode { get; init; } = BottomPanelMode.TextInput;
/// <summary>
/// Gets the queue of follow-up questions waiting for user answers. The head
/// (<c>[0]</c>) is the question currently being displayed; subsequent items
/// are dispatched in order as each is answered. While this queue is non-empty,
/// the next user submission is treated as the answer to the head question
/// instead of going to the agent runner's normal input handler.
/// </summary>
public IReadOnlyList<FollowUpQuestion> PendingQuestions { get; init; } = [];
/// <summary>
/// Gets the accumulated follow-up response messages collected during the
/// current agent turn — both direct <see cref="FollowUpMessage"/>s emitted
/// by observers and continuation results from answered questions. Consumed
/// by the runner via <see cref="IUXStateDriver.TakeFollowUpResponses"/>
/// before the next agent invocation.
/// </summary>
public IReadOnlyList<ChatMessage> AccumulatedFollowUpResponses { get; init; } = [];
// --- Text input (active in TextInput / Streaming modes) ---
/// <summary>Gets the prompt string for text input mode.</summary>
public string Prompt { get; init; } = "> ";
/// <summary>Gets the placeholder text shown when the input is empty.</summary>
public string Placeholder { get; init; } = "";
/// <summary>Gets the current input text being typed.</summary>
public string InputText { get; init; } = "";
/// <summary>Gets a value indicating whether input is enabled during streaming.</summary>
public bool InputEnabled { get; init; }
/// <summary>Gets the prompt to show during streaming when input is disabled.</summary>
public string StreamingPrompt { get; init; } = "(agent is running...)";
// --- List selection (active in ListSelection mode) ---
/// <summary>Gets the title text displayed above the list selection (for interactive prompts).</summary>
public string? ListSelectionTitle { get; init; }
/// <summary>Gets the list selection options.</summary>
public IReadOnlyList<string> ListSelectionOptions { get; init; } = [];
/// <summary>Gets the highlighted option index in list selection mode.</summary>
public int ListSelectionIndex { get; init; }
/// <summary>Gets the placeholder text for the custom text input option in the list.</summary>
public string? ListSelectionCustomTextPlaceholder { get; init; }
/// <summary>Gets the current text being typed into the list's custom text option.</summary>
public string ListSelectionCustomInputText { get; init; } = "";
/// <summary>Gets the highlight color for the active list item.</summary>
public ConsoleColor ListHighlightColor { get; init; } = ConsoleColor.Cyan;
// --- Scroll / output area ---
/// <summary>Gets the items rendered in the scroll-area. Each item is a pre-rendered
/// console string (may include ANSI escape sequences and newlines).</summary>
public IReadOnlyList<string> ScrollAreaContentItems { get; init; } = [];
/// <summary>Gets the queued input items to display above the rule. Each item is a
/// pre-rendered console string (may include ANSI escape sequences and newlines).</summary>
public IReadOnlyList<string> QueuedItems { get; init; } = [];
// --- Agent mode + status display ---
/// <summary>Gets the foreground color for the rule borders and mode label.</summary>
public ConsoleColor? ModeColor { get; init; }
/// <summary>Gets the current mode name displayed below the bottom rule (e.g. "plan").</summary>
public string? ModeText { get; init; }
/// <summary>Gets the help text displayed below the bottom rule (available commands).</summary>
public string? HelpText { get; init; }
/// <summary>Gets a value indicating whether the agent status spinner is visible.</summary>
public bool ShowSpinner { get; init; }
/// <summary>Gets the formatted token usage text to display in the status bar.</summary>
public string? UsageText { get; init; }
}
@@ -1,9 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
using Harness.Shared.Console.Commands;
using Harness.Shared.Console.Observers;
using Harness.ConsoleReactiveComponents;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console;
@@ -15,244 +13,58 @@ public static class HarnessConsole
{
/// <summary>
/// Runs an interactive console session with the specified agent.
/// Supports streaming output, tool call display, spinner animation,
/// optional planning UX with structured output, and the <c>/todos</c> command.
/// Constructs the reactive UI component and the <see cref="HarnessAgentRunner"/>,
/// wires them together, and awaits the component's <see cref="HarnessAppComponent.ShutdownTask"/>
/// (which completes when the user types <c>/exit</c>).
/// </summary>
/// <param name="agent">The agent to interact with.</param>
/// <param name="title">The title displayed in the console header.</param>
/// <param name="userPrompt">A short prompt to the user, displayed below the title.</param>
/// <param name="userPrompt">A short prompt to the user, displayed as a placeholder in the input area.</param>
/// <param name="options">Optional configuration options for the console session.</param>
public static async Task RunAgentAsync(AIAgent agent, string title, string userPrompt, HarnessConsoleOptions? options = null)
public static async Task RunAgentAsync(AIAgent agent, string userPrompt, HarnessConsoleOptions? options = null)
{
options ??= new();
if (options.EnablePlanningUx
&& (string.IsNullOrWhiteSpace(options.PlanningModeName) || string.IsNullOrWhiteSpace(options.ExecutionModeName)))
{
throw new ArgumentException(
"When EnablePlanningUx is true, both PlanningModeName and ExecutionModeName must be configured.",
nameof(options));
}
// Null means use defaults; an explicit (possibly empty) list means use exactly what was provided.
var observers = options.Observers
?? HarnessConsoleOptions.BuildDefaultObservers();
var commandHandlers = options.CommandHandlers
?? HarnessConsoleOptions.BuildDefaultCommandHandlers(agent, options.ModeColors);
var todoProvider = agent.GetService<TodoProvider>();
var modeProvider = agent.GetService<AgentModeProvider>();
var messageInjector = agent.GetService<MessageInjectingChatClient>();
var commandHandlers = new List<CommandHandler>
{
new TodoCommandHandler(todoProvider),
new ModeCommandHandler(modeProvider, options.ModeColors),
};
AgentSession session = await agent.CreateSessionAsync();
using var ux = new HarnessUXContainer(
using var component = new HarnessAppComponent(
placeholder: userPrompt,
initialMode: modeProvider?.GetMode(session),
inputEnabled: messageInjector is not null,
runnerFactory: ux => new HarnessAgentRunner(
agent: agent,
session: session,
modeProvider: modeProvider,
messageInjector: messageInjector,
commandHandlers: commandHandlers,
observers: observers,
ux: ux),
modeColors: options.ModeColors);
// Streaming-mode submissions are enqueued for injection; the queued display
// is then refreshed from the injector's current pending list.
ux.StreamingInputReceived += (sender, e) =>
// Trigger the initial render of the component now that state is seeded.
component.Render();
try
{
if (messageInjector is null)
{
return;
}
messageInjector.EnqueueMessages(session, [new ChatMessage(ChatRole.User, e.Text)]);
ux.ShowQueuedMessages(messageInjector.GetPendingMessages(session));
};
var commandHelp = commandHandlers
.Select(h => h.GetHelpText())
.Where(t => t is not null)
.Append("exit (quit)")!;
ux.Initialize(title, commandHelp!, messageInjector is not null);
string userInput = await ux.WaitForInputAsync();
while (!string.IsNullOrWhiteSpace(userInput) && !userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
await component.ShutdownTask.ConfigureAwait(false);
}
finally
{
ux.WriteUserInputEcho(userInput);
// Check command handlers first — first one to handle wins.
bool handled = false;
foreach (var handler in commandHandlers)
{
if (await handler.TryHandleAsync(userInput, session, ux).ConfigureAwait(false))
{
handled = true;
break;
}
}
if (!handled)
{
await RunAgentTurnAsync(agent, session, modeProvider, messageInjector, options, ux, userInput);
}
ux.CurrentMode = modeProvider?.GetMode(session);
userInput = await ux.WaitForInputAsync();
component.Deactivate();
}
ux.Deactivate();
System.Console.ResetColor();
System.Console.Write(AnsiEscapes.ResetScrollRegion);
System.Console.Write(AnsiEscapes.EraseEntireScreen);
System.Console.Write(AnsiEscapes.MoveCursor(1, 1));
System.Console.WriteLine("Goodbye!");
}
/// <summary>
/// Runs one or more agent invocations for a single user turn, using the current
/// observers. Re-invokes automatically for tool approvals and mode-driven follow-ups
/// (e.g., planning clarification loops).
/// </summary>
private static async Task RunAgentTurnAsync(
AIAgent agent,
AgentSession session,
AgentModeProvider? modeProvider,
MessageInjectingChatClient? messageInjector,
HarnessConsoleOptions options,
HarnessUXContainer ux,
string userInput)
{
IList<ChatMessage>? nextMessages = [new ChatMessage(ChatRole.User, userInput)];
IReadOnlyList<ChatMessage> lastPendingMessages = messageInjector?.GetPendingMessages(session) ?? [];
while (nextMessages is not null)
{
var observers = CreateObservers(options, modeProvider, session);
var runOptions = new AgentRunOptions();
foreach (var observer in observers)
{
observer.ConfigureRunOptions(runOptions);
}
ux.CurrentMode = modeProvider?.GetMode(session);
ux.BeginStreaming();
ux.BeginStreamingOutput();
try
{
await foreach (var update in agent.RunStreamingAsync(nextMessages, session, runOptions))
{
// Update mode color if the mode changed during streaming.
if (modeProvider is not null)
{
string currentMode = modeProvider.GetMode(session);
if (currentMode != ux.CurrentMode)
{
ux.CurrentMode = currentMode;
}
}
foreach (var content in update.Contents)
{
foreach (var observer in observers)
{
await observer.OnContentAsync(ux, content);
}
}
if (!string.IsNullOrEmpty(update.Text))
{
foreach (var observer in observers)
{
await observer.OnTextAsync(ux, update.Text);
}
}
SyncQueuedMessageDisplay(messageInjector, session, ux, ref lastPendingMessages);
}
}
catch (Exception ex)
{
await ux.WriteInfoLineAsync($"❌ Stream error: {ex.GetType().Name}:\n{ex}", ConsoleColor.Red);
}
// Final sync after streaming — messages may have been consumed during the last iteration.
SyncQueuedMessageDisplay(messageInjector, session, ux, ref lastPendingMessages);
// Stop spinner before observer completions (which may prompt for input).
ux.StopSpinner();
// Close the streaming output to provide visual separation from observer output.
await ux.EndStreamingOutputAsync();
var combinedMessages = new List<ChatMessage>();
bool hasObserverMessages = false;
foreach (var observer in observers)
{
var messages = await observer.OnStreamCompleteAsync(ux, agent, session, options);
if (messages is { Count: > 0 })
{
combinedMessages.AddRange(messages);
hasObserverMessages = true;
}
}
await ux.WriteNoTextWarningAsync(hasFollowUpMessages: hasObserverMessages);
ux.EndStreaming();
nextMessages = combinedMessages.Count > 0 ? combinedMessages : null;
}
}
/// <summary>
/// Synchronizes the queued items display with the message injector's pending messages.
/// Messages that have been consumed (drained by the service) are echoed to the output
/// area as regular user-input entries.
/// </summary>
private static void SyncQueuedMessageDisplay(
MessageInjectingChatClient? messageInjector,
AgentSession session,
HarnessUXContainer ux,
ref IReadOnlyList<ChatMessage> lastPendingMessages)
{
if (messageInjector is null)
{
return;
}
var pending = messageInjector.GetPendingMessages(session);
// If previously pending messages exceed current pending count, some were consumed.
int consumedCount = lastPendingMessages.Count - pending.Count;
for (int i = 0; i < consumedCount && i < lastPendingMessages.Count; i++)
{
string text = lastPendingMessages[i].Text ?? string.Empty;
ux.WriteUserInputEcho(text);
}
lastPendingMessages = pending;
ux.ShowQueuedMessages(pending);
}
private static List<ConsoleObserver> CreateObservers(HarnessConsoleOptions options, AgentModeProvider? modeProvider, AgentSession session)
{
var observers = new List<ConsoleObserver>
{
new ToolCallDisplayObserver(),
new ToolApprovalObserver(),
new ErrorDisplayObserver(),
new ReasoningDisplayObserver(),
new UsageDisplayObserver(options.MaxContextWindowTokens, options.MaxOutputTokens),
};
if (options.EnablePlanningUx
&& modeProvider is not null
&& string.Equals(modeProvider.GetMode(session), options.PlanningModeName, StringComparison.OrdinalIgnoreCase))
{
observers.Add(new PlanningOutputObserver(modeProvider));
}
else
{
observers.Add(new TextOutputObserver());
}
return observers;
}
}
@@ -1,5 +1,11 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.ObjectModel;
using Harness.Shared.Console.Commands;
using Harness.Shared.Console.Observers;
using Harness.Shared.Console.ToolFormatters;
using Microsoft.Agents.AI;
namespace Harness.Shared.Console;
/// <summary>
@@ -8,45 +14,120 @@ namespace Harness.Shared.Console;
public class HarnessConsoleOptions
{
/// <summary>
/// Gets or sets the optional maximum context window size in tokens.
/// When set, token usage is displayed as a percentage of the budget.
/// Gets or sets the list of console observers that participate in the agent response
/// streaming lifecycle. Use the factory methods on this class to create common observer sets.
/// When <see langword="null"/> (the default), a default set of observers is used.
/// Set to an empty list to disable all observers.
/// </summary>
public int? MaxContextWindowTokens { get; set; }
public IReadOnlyList<ConsoleObserver>? Observers { get; set; }
/// <summary>
/// Gets or sets the optional maximum output tokens.
/// Used with <see cref="MaxContextWindowTokens"/> to show input/output budget breakdown.
/// Gets or sets the list of command handlers to check before sending user input to the agent.
/// Use <see cref="BuildDefaultCommandHandlers"/> to create the default set.
/// When <see langword="null"/> (the default), a default set of handlers is used.
/// Set to an empty list to disable all command handlers.
/// </summary>
public int? MaxOutputTokens { get; set; }
public IReadOnlyList<CommandHandler>? CommandHandlers { get; set; }
/// <summary>
/// Gets or sets a value indicating whether the planning UX is enabled.
/// When <see langword="true"/> and the agent is in the mode specified by <see cref="PlanningModeName"/>,
/// the console uses structured output to present clarification questions and approval requests
/// instead of streaming free-form text.
/// The default mode-to-color mapping used when no custom <see cref="ModeColors"/> are provided.
/// </summary>
/// <value>Defaults to <see langword="false"/>.</value>
public bool EnablePlanningUx { get; set; }
/// <summary>
/// Gets or sets the name of the agent mode that activates the planning UX.
/// Must be set when <see cref="EnablePlanningUx"/> is <see langword="true"/>.
/// </summary>
public string? PlanningModeName { get; set; }
/// <summary>
/// Gets or sets the name of the agent mode to switch to when the user approves a plan.
/// Must be set when <see cref="EnablePlanningUx"/> is <see langword="true"/>.
/// </summary>
public string? ExecutionModeName { get; set; }
public static readonly IReadOnlyDictionary<string, ConsoleColor> DefaultModeColors = new ReadOnlyDictionary<string, ConsoleColor>(
new Dictionary<string, ConsoleColor>(StringComparer.OrdinalIgnoreCase)
{
["plan"] = ConsoleColor.Cyan,
["execute"] = ConsoleColor.Green,
});
/// <summary>
/// Gets or sets a mapping of agent mode names to console colors.
/// When a mode is not found in this dictionary, the default color (<see cref="ConsoleColor.Gray"/>) is used.
/// </summary>
public Dictionary<string, ConsoleColor> ModeColors { get; set; } = new(StringComparer.OrdinalIgnoreCase)
public Dictionary<string, ConsoleColor> ModeColors { get; set; } = new(DefaultModeColors, StringComparer.OrdinalIgnoreCase);
/// <summary>
/// Creates the default set of observers without planning support.
/// Includes tool call display, tool approval, error display, reasoning display,
/// usage display, and text output.
/// </summary>
/// <param name="maxContextWindowTokens">Optional maximum context window size in tokens for usage display.</param>
/// <param name="maxOutputTokens">Optional maximum output tokens for usage display.</param>
/// <param name="toolFormatters">Optional tool call formatters. When <see langword="null"/>,
/// each observer uses the default formatters from <see cref="ToolCallFormatter.BuildDefaultToolFormatters"/>.</param>
/// <returns>A list of observers for a standard (non-planning) console session.</returns>
public static List<ConsoleObserver> BuildDefaultObservers(
int? maxContextWindowTokens = null,
int? maxOutputTokens = null,
IReadOnlyList<ToolCallFormatter>? toolFormatters = null)
{
["plan"] = ConsoleColor.Cyan,
["execute"] = ConsoleColor.Green,
};
return
[
new ToolCallDisplayObserver(toolFormatters),
new ToolApprovalObserver(toolFormatters),
new ErrorDisplayObserver(),
new ReasoningDisplayObserver(),
new UsageDisplayObserver(maxContextWindowTokens, maxOutputTokens),
new TextOutputObserver(),
];
}
/// <summary>
/// Creates the default set of observers with planning support.
/// Includes a <see cref="PlanningOutputObserver"/> instead of <see cref="TextOutputObserver"/>.
/// </summary>
/// <param name="agent">The agent, used to resolve <see cref="AgentModeProvider"/>.</param>
/// <param name="planModeName">The mode name that represents the planning mode.</param>
/// <param name="executionModeName">The mode name to switch to when the user approves a plan.</param>
/// <param name="modeColors">Optional mode-to-color mapping for display.
/// Defaults to <see cref="DefaultModeColors"/> when <see langword="null"/>.</param>
/// <param name="maxContextWindowTokens">Optional maximum context window size in tokens for usage display.</param>
/// <param name="maxOutputTokens">Optional maximum output tokens for usage display.</param>
/// <param name="toolFormatters">Optional tool call formatters. When <see langword="null"/>,
/// each observer uses the default formatters from <see cref="ToolCallFormatter.BuildDefaultToolFormatters"/>.</param>
/// <returns>A list of observers for a planning-enabled console session.</returns>
public static List<ConsoleObserver> BuildObserversWithPlanning(
AIAgent agent,
string planModeName,
string executionModeName,
IReadOnlyDictionary<string, ConsoleColor>? modeColors = null,
int? maxContextWindowTokens = null,
int? maxOutputTokens = null,
IReadOnlyList<ToolCallFormatter>? toolFormatters = null)
{
var modeProvider = agent.GetService<AgentModeProvider>()
?? throw new InvalidOperationException("Planning requires an AgentModeProvider service on the agent.");
return
[
new ToolCallDisplayObserver(toolFormatters),
new ToolApprovalObserver(toolFormatters),
new ErrorDisplayObserver(),
new ReasoningDisplayObserver(),
new UsageDisplayObserver(maxContextWindowTokens, maxOutputTokens),
new PlanningOutputObserver(modeProvider, planModeName, executionModeName, modeColors ?? DefaultModeColors),
];
}
/// <summary>
/// Creates the default set of command handlers.
/// Includes exit, todo, and mode command handlers.
/// </summary>
/// <param name="agent">The agent, used to resolve <see cref="TodoProvider"/> and <see cref="AgentModeProvider"/>.</param>
/// <param name="modeColors">Optional mode-to-color mapping for the mode command display.
/// Defaults to <see cref="DefaultModeColors"/> when <see langword="null"/>.</param>
/// <returns>A list of command handlers for a standard console session.</returns>
public static List<CommandHandler> BuildDefaultCommandHandlers(
AIAgent agent,
IReadOnlyDictionary<string, ConsoleColor>? modeColors = null)
{
var todoProvider = agent.GetService<TodoProvider>();
var modeProvider = agent.GetService<AgentModeProvider>();
return
[
new ExitCommandHandler(),
new TodoCommandHandler(todoProvider),
new ModeCommandHandler(modeProvider, modeColors ?? DefaultModeColors),
];
}
}
@@ -0,0 +1,408 @@
// Copyright (c) Microsoft. All rights reserved.
using Harness.ConsoleReactiveComponents;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console;
/// <summary>
/// Default <see cref="IUXStateDriver"/> implementation. Owned by
/// <see cref="HarnessAppComponent"/>; mutates the component's state via a
/// <c>SetState</c>-style callback. Each public operation updates state and lets
/// the component's render-skip optimization handle the actual draw.
/// </summary>
internal sealed class HarnessConsoleUXStateDriver : IUXStateDriver
{
private readonly Func<HarnessAppComponentState> _getState;
private readonly Action<HarnessAppComponentState> _setState;
private readonly Action _requestShutdown;
private readonly IReadOnlyDictionary<string, ConsoleColor>? _modeColors;
private readonly List<string> _outputItems = [];
private readonly object _stateLock = new();
private OutputEntryType? _lastEntryType;
private bool _hasReceivedAnyText;
private OutputEntry? _currentStreamingEntry;
private int _currentStreamingEntryIndex = -1;
private string? _currentMode;
/// <summary>
/// Initializes a new instance of the <see cref="HarnessConsoleUXStateDriver"/> class.
/// </summary>
/// <param name="getState">Returns the component's current state.</param>
/// <param name="setState">Replaces the component's state and triggers a re-render.</param>
/// <param name="requestShutdown">Callback invoked when a command handler requests application shutdown.</param>
/// <param name="modeColors">Optional mapping of mode names to console colors.</param>
public HarnessConsoleUXStateDriver(
Func<HarnessAppComponentState> getState,
Action<HarnessAppComponentState> setState,
Action requestShutdown,
IReadOnlyDictionary<string, ConsoleColor>? modeColors = null)
{
this._getState = getState;
this._setState = setState;
this._requestShutdown = requestShutdown;
this._modeColors = modeColors;
this._currentMode = getState().ModeText;
}
/// <inheritdoc/>
public string? CurrentMode
{
get => this._currentMode;
set
{
this.UpdateState(s =>
{
this._currentMode = value;
return s with
{
ModeColor = ModeColors.Get(value, this._modeColors),
ModeText = value,
};
});
}
}
/// <inheritdoc/>
public void BeginStreaming() =>
this.UpdateState(s => s with
{
Mode = BottomPanelMode.Streaming,
ShowSpinner = true,
});
/// <inheritdoc/>
public void StopSpinner() =>
this.UpdateState(s => s with { ShowSpinner = false });
/// <inheritdoc/>
public void EndStreaming() =>
this.UpdateState(s => s with
{
Mode = BottomPanelMode.TextInput,
ShowSpinner = false,
});
/// <inheritdoc/>
public void BeginStreamingOutput()
{
lock (this._stateLock)
{
this._hasReceivedAnyText = false;
this._currentStreamingEntry = null;
this._currentStreamingEntryIndex = -1;
}
}
/// <inheritdoc/>
public void SetUsageText(string usageText) =>
this.UpdateState(s => s with { UsageText = usageText });
/// <inheritdoc/>
public void SetQueuedMessages(IReadOnlyList<ChatMessage> pending)
{
var newQueued = new List<string>(pending.Count);
foreach (var msg in pending)
{
string text = msg.Text ?? string.Empty;
newQueued.Add(RenderEntry($" 💬 {text}\n", ConsoleColor.DarkGray));
}
this.UpdateState(s => s with { QueuedItems = newQueued });
}
/// <inheritdoc/>
public void QueueFollowUpQuestions(IReadOnlyList<FollowUpQuestion> questions)
{
if (questions.Count == 0)
{
return;
}
this.UpdateState(s =>
{
bool wasEmpty = s.PendingQuestions.Count == 0;
var combined = new List<FollowUpQuestion>(s.PendingQuestions.Count + questions.Count);
combined.AddRange(s.PendingQuestions);
combined.AddRange(questions);
HarnessAppComponentState next = s with { PendingQuestions = combined };
if (wasEmpty)
{
next = this.ConfigureForHeadQuestion(next, combined[0]);
}
return next;
});
}
/// <inheritdoc/>
public void AddFollowUpResponse(ChatMessage response)
{
this.UpdateState(s =>
{
var combined = new List<ChatMessage>(s.AccumulatedFollowUpResponses.Count + 1);
combined.AddRange(s.AccumulatedFollowUpResponses);
combined.Add(response);
return s with { AccumulatedFollowUpResponses = combined };
});
}
/// <inheritdoc/>
public void AdvanceFollowUpQuestion()
{
this.UpdateState(s =>
{
if (s.PendingQuestions.Count == 0)
{
return s;
}
var remaining = s.PendingQuestions.Skip(1).ToList();
HarnessAppComponentState next = s with { PendingQuestions = remaining };
if (remaining.Count > 0)
{
return this.ConfigureForHeadQuestion(next, remaining[0]);
}
return next with
{
Mode = BottomPanelMode.TextInput,
ListSelectionOptions = [],
ListSelectionTitle = null,
ListSelectionCustomTextPlaceholder = null,
ListSelectionIndex = 0,
ListSelectionCustomInputText = "",
};
});
}
/// <inheritdoc/>
public IReadOnlyList<ChatMessage> TakeFollowUpResponses()
{
return this.UpdateState(s =>
{
IReadOnlyList<ChatMessage> responses = s.AccumulatedFollowUpResponses;
if (responses.Count == 0)
{
return (s, responses);
}
return (s with { AccumulatedFollowUpResponses = [] }, responses);
});
}
/// <summary>
/// Configures the bottom-panel display fields on the supplied state for the
/// given head question. For text questions, also writes the prompt as an
/// info line above the input row as a side effect.
/// </summary>
private HarnessAppComponentState ConfigureForHeadQuestion(HarnessAppComponentState state, FollowUpQuestion question)
{
if (question is ChoiceFollowUpQuestion choice)
{
return state with
{
Mode = BottomPanelMode.ListSelection,
ListSelectionOptions = choice.Choices.ToList(),
ListSelectionTitle = choice.Prompt,
ListSelectionCustomTextPlaceholder = choice.AllowCustomText ? "✏️ Type a custom response..." : null,
ListSelectionIndex = 0,
ListSelectionCustomInputText = "",
};
}
// Text question — prompt is rendered as an info line above the input row.
// We append entries and capture the scroll snapshot inline so the caller's
// single _setState picks up both the new output and the UI mode change.
ConsoleColor ruleColor = ModeColors.Get(this._currentMode, this._modeColors);
List<string> scrollSnapshot = this.AppendOutputEntriesAndSnapshot(
new OutputEntry(OutputEntryType.InfoLine, "\n", ruleColor),
new OutputEntry(OutputEntryType.InfoLine, $" {question.Prompt}", ruleColor));
return state with
{
Mode = BottomPanelMode.TextInput,
ListSelectionOptions = [],
ListSelectionTitle = null,
ListSelectionCustomTextPlaceholder = null,
ListSelectionIndex = 0,
ListSelectionCustomInputText = "",
ScrollAreaContentItems = scrollSnapshot,
};
}
/// <inheritdoc/>
public void WriteUserInputEcho(string text)
{
this.UpdateState(s =>
{
List<string> snapshot = this.AppendOutputEntriesAndSnapshot(new OutputEntry(
OutputEntryType.UserInput,
$"\nYou: {text}\n\n",
ConsoleColor.Green));
return s with { ScrollAreaContentItems = snapshot };
});
}
/// <inheritdoc/>
public Task WriteInfoAsync(string text, ConsoleColor? color = null) =>
this.WriteInfoCoreAsync(text, color, newLine: false);
/// <inheritdoc/>
public Task WriteInfoLineAsync(string text, ConsoleColor? color = null) =>
this.WriteInfoCoreAsync(text, color, newLine: true);
private Task WriteInfoCoreAsync(string text, ConsoleColor? color, bool newLine)
{
this.UpdateState(s =>
{
// Add a blank line separator when transitioning from streaming text or user input.
string prefix = this._lastEntryType is OutputEntryType.StreamingText or OutputEntryType.StreamFooter
? "\n "
: " ";
string fullText = newLine ? prefix + text + "\n\n" : prefix + text;
List<string> snapshot = this.AppendOutputEntriesAndSnapshot(new OutputEntry(
OutputEntryType.InfoLine,
fullText,
color ?? ModeColors.Get(this._currentMode, this._modeColors)));
return s with { ScrollAreaContentItems = snapshot };
});
return Task.CompletedTask;
}
/// <inheritdoc/>
public Task WriteTextAsync(string text, ConsoleColor? color = null)
{
this.UpdateState(s =>
{
this._lastEntryType = OutputEntryType.StreamingText;
this._hasReceivedAnyText = true;
ConsoleColor effectiveColor = color ?? ModeColors.Get(this._currentMode, this._modeColors);
if (this._currentStreamingEntry is not null
&& this._currentStreamingEntryIndex == this._outputItems.Count - 1)
{
// The streaming entry is still the last item — safe to replace in place.
this._currentStreamingEntry = this._currentStreamingEntry with
{
Text = this._currentStreamingEntry.Text + text,
};
this._outputItems[^1] = RenderEntry(this._currentStreamingEntry.Text, this._currentStreamingEntry.Color);
}
else
{
// Either the first text delta or other entries (tool calls, info lines)
// were appended after the previous streaming entry — start a fresh one.
const string Prefix = "\n";
this._currentStreamingEntry = new OutputEntry(OutputEntryType.StreamingText, Prefix + text, effectiveColor);
this._outputItems.Add(RenderEntry(this._currentStreamingEntry.Text, this._currentStreamingEntry.Color));
this._currentStreamingEntryIndex = this._outputItems.Count - 1;
}
return s with { ScrollAreaContentItems = new List<string>(this._outputItems) };
});
return Task.CompletedTask;
}
/// <inheritdoc/>
public Task EndStreamingOutputAsync()
{
this.UpdateState(s =>
{
if (this._hasReceivedAnyText)
{
this._outputItems.Add(RenderEntry("\n", null));
this._currentStreamingEntry = null;
this._lastEntryType = OutputEntryType.StreamFooter;
return s with { ScrollAreaContentItems = new List<string>(this._outputItems) };
}
return s;
});
return Task.CompletedTask;
}
/// <inheritdoc/>
public Task WriteNoTextWarningAsync(bool hasFollowUpActions)
{
if (!this._hasReceivedAnyText && !hasFollowUpActions)
{
this.UpdateState(s =>
{
List<string> snapshot = this.AppendOutputEntriesAndSnapshot(new OutputEntry(
OutputEntryType.StreamFooter,
" (no text response from agent)\n",
ConsoleColor.DarkYellow));
return s with { ScrollAreaContentItems = snapshot };
});
}
return Task.CompletedTask;
}
/// <summary>
/// Wraps the supplied text with ANSI foreground color escape sequences (or returns
/// the text unchanged when no color is specified). Output is appended to
/// <see cref="_outputItems"/> and consumed verbatim by <see cref="TextScrollPanel"/>
/// and <see cref="TextPanel"/>.
/// </summary>
private static string RenderEntry(string text, ConsoleColor? color) =>
color.HasValue
? $"{AnsiEscapes.SetForegroundColor(color.Value)}{text}{AnsiEscapes.ResetAttributes}"
: text;
private void UpdateState(Func<HarnessAppComponentState, HarnessAppComponentState> update)
{
lock (this._stateLock)
{
this._setState(update(this._getState()));
}
}
private T UpdateState<T>(Func<HarnessAppComponentState, (HarnessAppComponentState State, T Result)> update)
{
lock (this._stateLock)
{
var (newState, result) = update(this._getState());
this._setState(newState);
return result;
}
}
/// <summary>
/// Appends one or more output entries to the output list, updates
/// <see cref="_lastEntryType"/> to the last entry's type, and returns a
/// snapshot of <see cref="_outputItems"/>. Must be called inside a locked
/// context (e.g. within an <see cref="UpdateState"/> callback).
/// </summary>
private List<string> AppendOutputEntriesAndSnapshot(params OutputEntry[] entries)
{
this.AppendOutputEntriesCore(entries);
return new List<string>(this._outputItems);
}
private void AppendOutputEntriesCore(OutputEntry[] entries)
{
foreach (OutputEntry entry in entries)
{
this._outputItems.Add(RenderEntry(entry.Text, entry.Color));
}
if (entries.Length > 0)
{
this._lastEntryType = entries[^1].Type;
}
}
/// <inheritdoc/>
public void RequestShutdown() => this._requestShutdown();
}
@@ -1,478 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using Harness.ConsoleReactiveComponents;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console;
/// <summary>
/// Event arguments raised when the user submits text while the bottom panel is in
/// streaming mode (i.e. an agent turn is in progress).
/// </summary>
public sealed class StreamingInputReceivedEventArgs : EventArgs
{
/// <summary>
/// Initializes a new instance of the <see cref="StreamingInputReceivedEventArgs"/> class.
/// </summary>
/// <param name="text">The submitted text.</param>
public StreamingInputReceivedEventArgs(string text)
{
this.Text = text;
}
/// <summary>
/// Gets the submitted text.
/// </summary>
public string Text { get; }
}
/// <summary>
/// Façade over the harness UI: owns the <see cref="HarnessAppComponent"/>, manages
/// its props, dispatches input submissions, and provides the high-level read/write
/// operations used by observers, command handlers, and the harness loop.
/// </summary>
/// <remarks>
/// All callers interact with the UI exclusively through this class. The underlying
/// <see cref="HarnessAppComponent"/> and its props are an implementation detail and
/// must not be exposed.
/// </remarks>
public sealed class HarnessUXContainer : IDisposable
{
/// <summary>
/// The prompt displayed in the bottom-panel input area.
/// </summary>
private const string UserPrompt = "> ";
private readonly IReadOnlyDictionary<string, ConsoleColor>? _modeColors;
private readonly List<object> _outputItems = [];
private readonly HarnessAppComponent _appComponent;
private readonly object _outputLock = new();
private TaskCompletionSource<string>? _pendingInputTcs;
private OutputEntryType? _lastEntryType;
private bool _hasReceivedAnyText;
private OutputEntry? _currentStreamingEntry;
private string? _currentMode;
/// <summary>
/// Initializes a new instance of the <see cref="HarnessUXContainer"/> class.
/// </summary>
/// <param name="placeholder">Placeholder text shown when the input is empty.</param>
/// <param name="initialMode">The current agent mode, used to colour the rule and prompt.</param>
/// <param name="inputEnabled">Whether the bottom-panel input accepts keystrokes during streaming.</param>
/// <param name="modeColors">Optional mapping of mode names to console colors.</param>
public HarnessUXContainer(
string placeholder,
string? initialMode,
bool inputEnabled,
IReadOnlyDictionary<string, ConsoleColor>? modeColors = null)
{
this._modeColors = modeColors;
this._currentMode = initialMode;
this._appComponent = new HarnessAppComponent(RenderOutputEntry)
{
Props = new HarnessAppComponentProps
{
ScrollItems = this._outputItems,
Mode = BottomPanelMode.TextInput,
Prompt = UserPrompt,
Placeholder = placeholder,
ModeColor = ModeColors.Get(initialMode, modeColors),
ModeText = initialMode,
InputEnabled = inputEnabled,
},
};
this._appComponent.InputSubmitted += this.OnInputSubmitted;
}
/// <summary>
/// Raised when the user submits text while the bottom panel is in streaming mode.
/// Subscribers typically enqueue the text into a message-injecting chat client.
/// </summary>
public event EventHandler<StreamingInputReceivedEventArgs>? StreamingInputReceived;
/// <summary>
/// Gets or sets the current agent mode (e.g. "plan", "execute"). Updating this
/// also refreshes the rule colour and bottom-panel prompt to match the new mode.
/// </summary>
public string? CurrentMode
{
get => this._currentMode;
set
{
this._currentMode = value;
this._appComponent.Props = this._appComponent.Props! with
{
ModeColor = ModeColors.Get(value, this._modeColors),
ModeText = value,
};
this._appComponent.Render();
}
}
/// <summary>
/// Performs the initial screen clear, sets the help text in the mode-and-help bar,
/// and adds the title to the output area.
/// </summary>
/// <param name="title">The title displayed in the console header.</param>
/// <param name="commandHelpTexts">The command help strings displayed in the mode-and-help bar.</param>
/// <param name="messageInjectionActive">Whether streaming-time message injection is enabled.</param>
public void Initialize(string title, IEnumerable<string> commandHelpTexts, bool messageInjectionActive)
{
// Set the help text on the mode-and-help bar (persists below the rule).
this._appComponent.Props = this._appComponent.Props! with
{
HelpText = string.Join(", ", commandHelpTexts),
ModeText = this._currentMode,
};
System.Console.Write(AnsiEscapes.EraseEntireScreen);
System.Console.Write(AnsiEscapes.EraseScrollbackBuffer);
this._appComponent.Render();
this.AppendOutputEntries(
new OutputEntry(OutputEntryType.InfoLine, $"=== {title} ===\n", ConsoleColor.White),
new OutputEntry(OutputEntryType.InfoLine, "\n"));
}
/// <summary>
/// Restores the cursor and exits the alternate screen, ending the interactive UI.
/// </summary>
public void Deactivate() => this._appComponent.Deactivate();
/// <summary>
/// Switches the bottom panel to streaming mode and starts the spinner.
/// </summary>
public void BeginStreaming()
{
this._appComponent.Props = this._appComponent.Props! with
{
Mode = BottomPanelMode.Streaming,
ShowSpinner = true,
};
this._appComponent.Render();
}
/// <summary>
/// Stops the spinner without leaving streaming mode. Use between the end of the
/// stream and any observer-driven prompts (e.g. tool approvals).
/// </summary>
public void StopSpinner()
{
this._appComponent.Props = this._appComponent.Props! with { ShowSpinner = false };
this._appComponent.Render();
}
/// <summary>
/// Switches the bottom panel back to text-input mode and stops the spinner.
/// </summary>
public void EndStreaming()
{
this._appComponent.Props = this._appComponent.Props! with
{
Mode = BottomPanelMode.TextInput,
ShowSpinner = false,
};
this._appComponent.Render();
}
/// <summary>
/// Resets per-turn streaming bookkeeping in preparation for a new agent turn.
/// </summary>
public void BeginStreamingOutput()
{
this._hasReceivedAnyText = false;
this._currentStreamingEntry = null;
}
/// <summary>
/// Sets the formatted usage text shown on the agent status bar.
/// </summary>
public void SetUsageText(string usageText)
{
this._appComponent.Props = this._appComponent.Props! with { UsageText = usageText };
this._appComponent.Render();
}
/// <summary>
/// Clears the usage text from the agent status bar.
/// </summary>
public void ClearUsageText()
{
this._appComponent.Props = this._appComponent.Props! with { UsageText = null };
this._appComponent.Render();
}
/// <summary>
/// Replaces the queued-message display with one entry per pending message.
/// </summary>
public void ShowQueuedMessages(IReadOnlyList<ChatMessage> pending)
{
var newQueued = new List<object>(pending.Count);
foreach (var msg in pending)
{
string text = msg.Text ?? string.Empty;
newQueued.Add(new OutputEntry(OutputEntryType.UserInput, $" 💬 {text}\n", ConsoleColor.DarkGray));
}
this._appComponent.Props = this._appComponent.Props! with { QueuedItems = newQueued };
this._appComponent.Render();
}
/// <summary>
/// Echoes a submitted user input as a regular user-input entry in the output area,
/// using the current mode-aware prompt prefix.
/// </summary>
/// <param name="text">The user-entered text.</param>
public void WriteUserInputEcho(string text)
{
this.AppendOutputEntries(new OutputEntry(
OutputEntryType.UserInput,
$"\nYou: {text}\n",
ConsoleColor.Green));
}
/// <summary>
/// Writes informational output as an output entry, without a trailing newline.
/// </summary>
public Task WriteInfoAsync(string text, ConsoleColor? color = null) =>
this.WriteInfoCoreAsync(text, color, newLine: false);
/// <summary>
/// Writes informational output as an output entry, followed by a newline.
/// </summary>
public Task WriteInfoLineAsync(string text, ConsoleColor? color = null) =>
this.WriteInfoCoreAsync(text, color, newLine: true);
private Task WriteInfoCoreAsync(string text, ConsoleColor? color, bool newLine)
{
// Add a blank line separator when transitioning from streaming text or user input.
string prefix = this._lastEntryType is OutputEntryType.StreamingText or OutputEntryType.StreamFooter
? "\n\n "
: " ";
string fullText = newLine ? prefix + text + "\n" : prefix + text;
this.AppendOutputEntries(new OutputEntry(
OutputEntryType.InfoLine,
fullText,
color ?? ModeColors.Get(this.CurrentMode, this._modeColors)));
return Task.CompletedTask;
}
/// <summary>
/// Writes streaming text output from the agent. Successive calls accumulate into a
/// single streaming entry that is re-rendered by the text panel.
/// </summary>
public Task WriteTextAsync(string text, ConsoleColor? color = null)
{
lock (this._outputLock)
{
this._lastEntryType = OutputEntryType.StreamingText;
this._hasReceivedAnyText = true;
ConsoleColor effectiveColor = color ?? ModeColors.Get(this.CurrentMode, this._modeColors);
if (this._currentStreamingEntry is not null)
{
this._currentStreamingEntry = this._currentStreamingEntry with
{
Text = this._currentStreamingEntry.Text + text,
};
this._outputItems[^1] = this._currentStreamingEntry;
}
else
{
const string Prefix = "\n";
this._currentStreamingEntry = new OutputEntry(OutputEntryType.StreamingText, Prefix + text, effectiveColor);
this._outputItems.Add(this._currentStreamingEntry);
}
this._appComponent.Props = this._appComponent.Props! with
{
ScrollItems = new List<object>(this._outputItems),
};
}
this._appComponent.Render();
return Task.CompletedTask;
}
/// <summary>
/// Writes a blank-line separator to visually close the streaming output section.
/// Call before observer completions so their output is visually separated.
/// </summary>
public Task EndStreamingOutputAsync()
{
lock (this._outputLock)
{
this._outputItems.Add(new OutputEntry(OutputEntryType.StreamFooter, "\n"));
this._currentStreamingEntry = null;
this._lastEntryType = OutputEntryType.StreamFooter;
this._appComponent.Props = this._appComponent.Props! with
{
ScrollItems = new List<object>(this._outputItems),
};
}
this._appComponent.Render();
return Task.CompletedTask;
}
/// <summary>
/// Shows a "(no text response from agent)" warning if no text was received
/// and no observer produced follow-up messages. Call after observer completions.
/// </summary>
/// <param name="hasFollowUpMessages">Whether any observer produced follow-up messages.</param>
public Task WriteNoTextWarningAsync(bool hasFollowUpMessages)
{
if (!this._hasReceivedAnyText && !hasFollowUpMessages)
{
this.AppendOutputEntries(new OutputEntry(
OutputEntryType.StreamFooter,
" (no text response from agent)\n",
ConsoleColor.DarkYellow));
}
return Task.CompletedTask;
}
/// <summary>
/// Reads a line of input from the user. If <paramref name="prompt"/> is supplied
/// it is rendered as an info line above the input row before reading.
/// </summary>
public async Task<string?> ReadLineAsync(string? prompt = null, ConsoleColor? promptColor = null)
{
if (prompt is not null)
{
ConsoleColor ruleColor = ModeColors.Get(this.CurrentMode, this._modeColors);
this.AppendOutputEntries(
new OutputEntry(OutputEntryType.InfoLine, "\n", ruleColor),
new OutputEntry(OutputEntryType.InfoLine, $" {prompt}", promptColor ?? ruleColor));
}
this._appComponent.Props = this._appComponent.Props! with { Mode = BottomPanelMode.TextInput };
this._appComponent.Render();
string input = await this.WaitForInputAsync();
this.AppendOutputEntries(new OutputEntry(
OutputEntryType.UserInput,
$"\nYou: {input}\n",
ConsoleColor.Green));
return input;
}
/// <summary>
/// Presents a selection prompt with the given choices and waits for the user's
/// selection. The title is displayed above the list in the bottom panel. After
/// selection the bottom panel is restored to text-input mode and both the question
/// and selection are echoed in the output area.
/// </summary>
public async Task<string> ReadSelectionAsync(string title, IList<string> choices)
{
this._appComponent.Props = this._appComponent.Props! with
{
Mode = BottomPanelMode.ListSelection,
Items = choices.ToList(),
ListTitle = title,
ListCustomTextPlaceholder = "✏️ Type a custom response...",
};
this._appComponent.Render();
string selection = await this.WaitForInputAsync();
this._appComponent.Props = this._appComponent.Props with { Mode = BottomPanelMode.TextInput };
this.AppendOutputEntries(
new OutputEntry(
OutputEntryType.InfoLine,
$"\n {title}\n",
ModeColors.Get(this.CurrentMode, this._modeColors)),
new OutputEntry(
OutputEntryType.UserInput,
$"\nYou: {selection}\n",
ConsoleColor.Green));
return selection;
}
/// <summary>
/// Awaits the next non-streaming user input submission.
/// </summary>
public Task<string> WaitForInputAsync()
{
this._pendingInputTcs = new TaskCompletionSource<string>(TaskCreationOptions.RunContinuationsAsynchronously);
return this._pendingInputTcs.Task;
}
private void OnInputSubmitted(object? sender, InputSubmittedEventArgs e)
{
if (e.Mode == BottomPanelMode.Streaming)
{
this.StreamingInputReceived?.Invoke(this, new StreamingInputReceivedEventArgs(e.Text));
}
else
{
var waiter = this._pendingInputTcs;
this._pendingInputTcs = null;
waiter?.TrySetResult(e.Text);
}
}
/// <inheritdoc/>
public void Dispose()
{
this._appComponent.InputSubmitted -= this.OnInputSubmitted;
this._appComponent.Deactivate();
this._appComponent.Dispose();
}
/// <summary>
/// Renders an <see cref="OutputEntry"/> to a string with ANSI color codes.
/// Used as the render delegate for the <see cref="HarnessAppComponent"/>.
/// </summary>
private static string RenderOutputEntry(object item)
{
if (item is not OutputEntry entry)
{
return item?.ToString() ?? string.Empty;
}
if (entry.Color.HasValue)
{
return $"{AnsiEscapes.SetForegroundColor(entry.Color.Value)}{entry.Text}{AnsiEscapes.ResetAttributes}";
}
return entry.Text;
}
/// <summary>
/// Appends one or more output entries to the output list under lock,
/// updates <see cref="_lastEntryType"/> to the last entry's type, and renders.
/// </summary>
private void AppendOutputEntries(params OutputEntry[] entries)
{
lock (this._outputLock)
{
foreach (OutputEntry entry in entries)
{
this._outputItems.Add(entry);
}
if (entries.Length > 0)
{
this._lastEntryType = entries[^1].Type;
}
this._appComponent.Props = this._appComponent.Props! with
{
ScrollItems = new List<object>(this._outputItems),
};
}
this._appComponent.Render();
}
}
@@ -0,0 +1,120 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console;
/// <summary>
/// Abstraction over the harness UI state. All callers (observers, command handlers,
/// the agent runner) interact with the UI exclusively through this interface, which
/// internally translates each operation into a <c>SetState</c> call on the underlying
/// reactive component.
/// </summary>
/// <remarks>
/// This interface is intentionally narrow: it does not expose blocking input methods.
/// The agent runner orchestrates input flow via <see cref="FollowUpQuestion"/>
/// objects returned from observers.
/// </remarks>
public interface IUXStateDriver
{
/// <summary>
/// Gets or sets the current agent mode (e.g. "plan", "execute"). Setting also
/// refreshes the rule colour and bottom-panel prompt to match the new mode.
/// </summary>
string? CurrentMode { get; set; }
/// <summary>
/// Echoes a submitted user input as a regular user-input entry in the output area.
/// </summary>
void WriteUserInputEcho(string text);
/// <summary>
/// Writes informational output as an output entry, without a trailing newline.
/// </summary>
Task WriteInfoAsync(string text, ConsoleColor? color = null);
/// <summary>
/// Writes informational output as an output entry, followed by a newline.
/// </summary>
Task WriteInfoLineAsync(string text, ConsoleColor? color = null);
/// <summary>
/// Writes streaming text output from the agent. Successive calls accumulate into a
/// single streaming entry that is re-rendered by the text panel.
/// </summary>
Task WriteTextAsync(string text, ConsoleColor? color = null);
/// <summary>
/// Writes a blank-line separator to visually close the streaming output section.
/// </summary>
Task EndStreamingOutputAsync();
/// <summary>
/// Shows a "(no text response from agent)" warning if no text was received
/// and no observer produced follow-up actions.
/// </summary>
Task WriteNoTextWarningAsync(bool hasFollowUpActions);
/// <summary>
/// Switches the bottom panel to streaming mode and starts the spinner.
/// </summary>
void BeginStreaming();
/// <summary>
/// Stops the spinner without leaving streaming mode.
/// </summary>
void StopSpinner();
/// <summary>
/// Switches the bottom panel back to text-input mode and stops the spinner.
/// </summary>
void EndStreaming();
/// <summary>
/// Resets per-turn streaming bookkeeping in preparation for a new agent turn.
/// </summary>
void BeginStreamingOutput();
/// <summary>
/// Sets the formatted usage text shown on the agent status bar.
/// </summary>
void SetUsageText(string usageText);
/// <summary>
/// Replaces the queued-message display with one entry per pending message.
/// </summary>
void SetQueuedMessages(IReadOnlyList<ChatMessage> pending);
/// <summary>
/// Appends the supplied questions to the pending follow-up question queue in
/// component state. If the queue was empty, the bottom-panel display is
/// reconfigured to present the new head question.
/// </summary>
void QueueFollowUpQuestions(IReadOnlyList<FollowUpQuestion> questions);
/// <summary>
/// Appends a message to the accumulated follow-up response list in component state.
/// Called by the runner for direct <see cref="FollowUpMessage"/> outputs and by
/// the component when a question's continuation produces a response.
/// </summary>
void AddFollowUpResponse(ChatMessage response);
/// <summary>
/// Pops the head of the pending follow-up question queue. Reconfigures the
/// bottom-panel display for the new head, or restores the default text-input
/// mode if the queue is now empty.
/// </summary>
void AdvanceFollowUpQuestion();
/// <summary>
/// Returns the current accumulated follow-up responses and clears them in state.
/// Called by the runner immediately before invoking the next agent turn.
/// </summary>
IReadOnlyList<ChatMessage> TakeFollowUpResponses();
/// <summary>
/// Signals that the application should shut down. Completes the shutdown task
/// on the owning component.
/// </summary>
void RequestShutdown();
}
@@ -18,36 +18,41 @@ public abstract class ConsoleObserver
/// Override to set options such as <see cref="AgentRunOptions.ResponseFormat"/>.
/// </summary>
/// <param name="options">The run options to configure.</param>
public virtual void ConfigureRunOptions(AgentRunOptions options)
/// <param name="agent">The agent being interacted with.</param>
/// <param name="session">The current agent session.</param>
public virtual void ConfigureRunOptions(AgentRunOptions options, AIAgent agent, AgentSession session)
{
}
/// <summary>
/// Called for each <see cref="AIContent"/> item in the response stream.
/// </summary>
/// <param name="ux">The harness UX container, used for rendering output and interacting with the user.</param>
/// <param name="ux">The UX state driver, used for rendering output.</param>
/// <param name="content">The content item from the stream.</param>
public virtual Task OnContentAsync(HarnessUXContainer ux, AIContent content) => Task.CompletedTask;
/// <param name="agent">The agent being interacted with.</param>
/// <param name="session">The current agent session.</param>
public virtual Task OnContentAsync(IUXStateDriver ux, AIContent content, AIAgent agent, AgentSession session) => Task.CompletedTask;
/// <summary>
/// Called for each text update in the response stream.
/// </summary>
/// <param name="ux">The harness UX container, used for rendering output and interacting with the user.</param>
/// <param name="ux">The UX state driver, used for rendering output.</param>
/// <param name="text">The text from the update.</param>
public virtual Task OnTextAsync(HarnessUXContainer ux, string text) => Task.CompletedTask;
/// <summary>
/// Called after the response stream completes. Returns messages to include in the
/// next agent invocation, or <see langword="null"/> if no re-invocation is needed.
/// </summary>
/// <param name="ux">The harness UX container, used for rendering output and interacting with the user.</param>
/// <param name="agent">The agent being interacted with.</param>
/// <param name="session">The current agent session.</param>
/// <param name="options">The console options.</param>
/// <returns>Messages to send to the agent, or <see langword="null"/> if no action is needed.</returns>
public virtual Task<IList<ChatMessage>?> OnStreamCompleteAsync(
HarnessUXContainer ux,
public virtual Task OnTextAsync(IUXStateDriver ux, string text, AIAgent agent, AgentSession session) => Task.CompletedTask;
/// <summary>
/// Called after the response stream completes. Returns a heterogeneous list of
/// follow-up actions (questions to ask the user, and/or messages to add directly to
/// the next agent invocation), or <see langword="null"/> if no follow-up is needed.
/// </summary>
/// <param name="ux">The UX state driver, used for rendering output.</param>
/// <param name="agent">The agent being interacted with.</param>
/// <param name="session">The current agent session.</param>
/// <returns>Follow-up actions to process after the stream completes, or <see langword="null"/>.</returns>
public virtual Task<IList<FollowUpAction>?> OnStreamCompleteAsync(
IUXStateDriver ux,
AIAgent agent,
AgentSession session,
HarnessConsoleOptions options) => Task.FromResult<IList<ChatMessage>?>(null);
AgentSession session) => Task.FromResult<IList<FollowUpAction>?>(null);
}
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.Observers;
@@ -7,10 +8,10 @@ namespace Harness.Shared.Console.Observers;
/// <summary>
/// Displays error content (❌) from the response stream.
/// </summary>
internal sealed class ErrorDisplayObserver : ConsoleObserver
public sealed class ErrorDisplayObserver : ConsoleObserver
{
/// <inheritdoc/>
public override async Task OnContentAsync(HarnessUXContainer ux, AIContent content)
public override async Task OnContentAsync(IUXStateDriver ux, AIContent content, AIAgent agent, AgentSession session)
{
if (content is ErrorContent errorContent)
{
@@ -2,51 +2,77 @@
using System.Text;
using System.Text.Json;
using Harness.ConsoleReactiveComponents;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.Observers;
/// <summary>
/// Planning observer that configures structured output, collects streamed text,
/// and deserializes it as a <see cref="PlanningResponse"/>. Renders clarification
/// questions and approval prompts, and manages mode switching when the user approves a plan.
/// Planning observer that is mode-aware: in planning mode it configures structured
/// JSON output, collects streamed text, and deserializes it as a <see cref="PlanningResponse"/>;
/// in execution mode it passes text straight through to <see cref="IUXStateDriver.WriteTextAsync"/>
/// for live streaming display.
/// </summary>
internal sealed class PlanningOutputObserver : ConsoleObserver
public sealed class PlanningOutputObserver : ConsoleObserver
{
private readonly StringBuilder _textCollector = new();
private readonly AgentModeProvider _modeProvider;
private readonly string _planModeName;
private readonly string _executionModeName;
private readonly IReadOnlyDictionary<string, ConsoleColor>? _modeColors;
/// <summary>
/// Initializes a new instance of the <see cref="PlanningOutputObserver"/> class.
/// </summary>
/// <param name="modeProvider">The mode provider for switching modes on approval.</param>
public PlanningOutputObserver(AgentModeProvider modeProvider)
/// <param name="planModeName">The mode name that represents the planning mode.</param>
/// <param name="executionModeName">The mode name to switch to when the user approves a plan.</param>
/// <param name="modeColors">Optional mode-to-color mapping for display.</param>
public PlanningOutputObserver(AgentModeProvider modeProvider, string planModeName, string executionModeName, IReadOnlyDictionary<string, ConsoleColor>? modeColors = null)
{
this._modeProvider = modeProvider;
this._planModeName = planModeName;
this._executionModeName = executionModeName;
this._modeColors = modeColors;
}
/// <inheritdoc/>
public override void ConfigureRunOptions(AgentRunOptions options)
public override void ConfigureRunOptions(AgentRunOptions options, AIAgent agent, AgentSession session)
{
options.ResponseFormat = ChatResponseFormat.ForJsonSchema<PlanningResponse>();
if (this.IsPlanningMode(this._modeProvider.GetMode(session)))
{
options.ResponseFormat = ChatResponseFormat.ForJsonSchema<PlanningResponse>();
}
}
/// <inheritdoc/>
public override Task OnTextAsync(HarnessUXContainer ux, string text)
public override Task OnTextAsync(IUXStateDriver ux, string text, AIAgent agent, AgentSession session)
{
// Collect text silently instead of displaying it.
this._textCollector.Append(text);
return Task.CompletedTask;
if (this.IsPlanningMode(ux.CurrentMode))
{
// Planning mode: collect text silently for JSON parsing after the stream.
this._textCollector.Append(text);
return Task.CompletedTask;
}
// Execution mode: stream text directly to the console.
return ux.WriteTextAsync(text);
}
/// <inheritdoc/>
public override async Task<IList<ChatMessage>?> OnStreamCompleteAsync(
HarnessUXContainer ux,
public override async Task<IList<FollowUpAction>?> OnStreamCompleteAsync(
IUXStateDriver ux,
AIAgent agent,
AgentSession session,
HarnessConsoleOptions options)
AgentSession session)
{
if (!this.IsPlanningMode(ux.CurrentMode))
{
// Execution mode: text was already streamed live; nothing to parse.
this._textCollector.Clear();
return null;
}
// Read collected text from our stream observation.
string collectedText = this._textCollector.ToString();
this._textCollector.Clear();
@@ -75,10 +101,9 @@ internal sealed class PlanningOutputObserver : ConsoleObserver
return null;
}
// Render based on response type.
if (planningResponse.Type == PlanningResponseType.Clarification)
{
return AsUserMessages(await this.RenderClarificationsAndCollectResponsesAsync(ux, planningResponse));
return BuildClarificationActions(planningResponse);
}
if (planningResponse.Type == PlanningResponseType.Approval)
@@ -90,67 +115,87 @@ internal sealed class PlanningOutputObserver : ConsoleObserver
return null;
}
string response = await this.RenderApprovalAndCollectResponseAsync(ux, question, options);
if (response == "Approved")
{
this._modeProvider.SetMode(session, options.ExecutionModeName!);
await ux.WriteInfoLineAsync($"✅ Switched to {options.ExecutionModeName} mode.",
ModeColors.Get(options.ExecutionModeName, options.ModeColors));
}
return AsUserMessages(response);
return new List<FollowUpAction> { this.BuildApprovalAction(question, session) };
}
await ux.WriteInfoLineAsync($"(unexpected response type: {planningResponse.Type})", ConsoleColor.DarkYellow);
return null;
}
private static IList<ChatMessage>? AsUserMessages(string? text) =>
text is not null ? [new ChatMessage(ChatRole.User, text)] : null;
private async Task<string?> RenderClarificationsAndCollectResponsesAsync(HarnessUXContainer ux, PlanningResponse response)
private static List<FollowUpAction> BuildClarificationActions(PlanningResponse response)
{
var answers = new List<string>();
var actions = new List<FollowUpAction>(response.Questions.Count);
foreach (var question in response.Questions)
{
string? answer;
string prompt = question.Message;
async Task<ChatMessage?> Continuation(string answer, IUXStateDriver ux)
{
if (string.IsNullOrWhiteSpace(answer))
{
string noAnswer = $"🔹 {prompt}\n └─ {AnsiEscapes.SetForegroundColor(ConsoleColor.DarkGray)}(no answer){AnsiEscapes.ResetAttributes}";
await ux.WriteInfoLineAsync(noAnswer, ConsoleColor.Gray).ConfigureAwait(false);
return null;
}
string formatted = $"🔹 {prompt}\n └─ {AnsiEscapes.SetForegroundColor(ConsoleColor.Green)}{answer}{AnsiEscapes.ResetAttributes}";
await ux.WriteInfoLineAsync(formatted, ConsoleColor.Gray).ConfigureAwait(false);
return new ChatMessage(ChatRole.User, $"Q: {prompt}\nA: {answer}");
}
if (question.Choices is { Count: > 0 })
{
answer = await ux.ReadSelectionAsync(
question.Message,
question.Choices);
actions.Add(new ChoiceFollowUpQuestion(
Prompt: prompt,
Choices: question.Choices,
AllowCustomText: true,
Continuation: Continuation));
}
else
{
answer = (await ux.ReadLineAsync(question.Message))?.Trim();
}
if (!string.IsNullOrWhiteSpace(answer))
{
answers.Add($"Q: {question.Message}\nA: {answer}");
actions.Add(new TextFollowUpQuestion(
Prompt: prompt,
Continuation: Continuation));
}
}
return answers.Count > 0 ? string.Join("\n\n", answers) : null;
return actions;
}
private async Task<string> RenderApprovalAndCollectResponseAsync(HarnessUXContainer ux, PlanningQuestion question, HarnessConsoleOptions options)
private ChoiceFollowUpQuestion BuildApprovalAction(PlanningQuestion question, AgentSession session)
{
var choices = new List<string>
{
"Approve and switch to execute mode",
};
const string ApproveOption = "Approve and switch to execute mode";
var choices = new List<string> { ApproveOption };
string selection = await ux.ReadSelectionAsync(question.Message, choices);
return new ChoiceFollowUpQuestion(
Prompt: question.Message,
Choices: choices,
AllowCustomText: true,
Continuation: async (selection, ux) =>
{
string formatted = $"🔹 {question.Message}\n └─ {AnsiEscapes.SetForegroundColor(ConsoleColor.Green)}{selection}{AnsiEscapes.ResetAttributes}";
await ux.WriteInfoLineAsync(formatted, ConsoleColor.Gray).ConfigureAwait(false);
if (selection == choices[0])
{
return "Approved";
}
if (selection == ApproveOption)
{
this._modeProvider.SetMode(session, this._executionModeName);
await ux.WriteInfoLineAsync(
$"✅ Switched to {this._executionModeName} mode.",
ModeColors.Get(this._executionModeName, this._modeColors)).ConfigureAwait(false);
return new ChatMessage(ChatRole.User, "Approved");
}
// Custom freeform input — treat as suggested changes.
return selection;
// Custom freeform input — treat as suggested changes.
return new ChatMessage(ChatRole.User, selection);
});
}
/// <summary>
/// Returns <see langword="true"/> when the current mode matches the configured plan mode name.
/// A <see langword="null"/> mode (no mode provider) is also treated as planning mode.
/// </summary>
private bool IsPlanningMode(string? currentMode) =>
currentMode is null || string.Equals(currentMode, this._planModeName, StringComparison.OrdinalIgnoreCase);
}
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.Observers;
@@ -7,10 +8,10 @@ namespace Harness.Shared.Console.Observers;
/// <summary>
/// Displays reasoning content in dark magenta from the response stream.
/// </summary>
internal sealed class ReasoningDisplayObserver : ConsoleObserver
public sealed class ReasoningDisplayObserver : ConsoleObserver
{
/// <inheritdoc/>
public override async Task OnContentAsync(HarnessUXContainer ux, AIContent content)
public override async Task OnContentAsync(IUXStateDriver ux, AIContent content, AIAgent agent, AgentSession session)
{
if (content is TextReasoningContent reasoning && !string.IsNullOrEmpty(reasoning.Text))
{
@@ -1,15 +1,17 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI;
namespace Harness.Shared.Console.Observers;
/// <summary>
/// Streams agent text output directly to the console.
/// Used in normal (non-planning) mode.
/// </summary>
internal sealed class TextOutputObserver : ConsoleObserver
public sealed class TextOutputObserver : ConsoleObserver
{
/// <inheritdoc/>
public override async Task OnTextAsync(HarnessUXContainer ux, string text)
public override async Task OnTextAsync(IUXStateDriver ux, string text, AIAgent agent, AgentSession session)
{
await ux.WriteTextAsync(text);
}
@@ -1,5 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
using Harness.ConsoleReactiveComponents;
using Harness.Shared.Console.ToolFormatters;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
@@ -7,86 +9,103 @@ namespace Harness.Shared.Console.Observers;
/// <summary>
/// Collects <see cref="ToolApprovalRequestContent"/> items during the response stream,
/// displays approval-needed notifications inline, and prompts the user for approval
/// decisions after the stream completes.
/// displays approval-needed notifications inline, and after the stream completes returns
/// one <see cref="ChoiceFollowUpQuestion"/> per pending approval request. Each question's
/// continuation produces a separate <see cref="ChatMessage"/> carrying the approval
/// response content.
/// </summary>
internal sealed class ToolApprovalObserver : ConsoleObserver
public sealed class ToolApprovalObserver : ConsoleObserver
{
private readonly List<ToolApprovalRequestContent> _approvalRequests = [];
private readonly IReadOnlyList<ToolCallFormatter> _formatters;
/// <summary>
/// Initializes a new instance of the <see cref="ToolApprovalObserver"/> class.
/// </summary>
/// <param name="formatters">Optional list of tool formatters. When <see langword="null"/>,
/// the default formatters from <see cref="ToolCallFormatter.BuildDefaultToolFormatters"/> are used.</param>
public ToolApprovalObserver(IReadOnlyList<ToolCallFormatter>? formatters = null)
{
this._formatters = formatters ?? ToolCallFormatter.BuildDefaultToolFormatters();
}
/// <inheritdoc/>
public override async Task OnContentAsync(HarnessUXContainer ux, AIContent content)
public override async Task OnContentAsync(IUXStateDriver ux, AIContent content, AIAgent agent, AgentSession session)
{
if (content is ToolApprovalRequestContent approvalRequest)
{
this._approvalRequests.Add(approvalRequest);
string toolName = approvalRequest.ToolCall is FunctionCallContent fc
? ToolCallFormatter.Format(fc)
? ToolCallFormatter.Format(this._formatters, fc)
: approvalRequest.ToolCall?.ToString() ?? "unknown";
await ux.WriteInfoLineAsync($"⚠️ Approval needed: {toolName}", ConsoleColor.Yellow);
}
}
/// <inheritdoc/>
public override async Task<IList<ChatMessage>?> OnStreamCompleteAsync(
HarnessUXContainer ux,
public override Task<IList<FollowUpAction>?> OnStreamCompleteAsync(
IUXStateDriver ux,
AIAgent agent,
AgentSession session,
HarnessConsoleOptions options)
AgentSession session)
{
if (this._approvalRequests.Count == 0)
{
return null;
return Task.FromResult<IList<FollowUpAction>?>(null);
}
var actions = new List<FollowUpAction>(this._approvalRequests.Count);
foreach (var request in this._approvalRequests)
{
actions.Add(this.BuildApprovalQuestion(request));
}
var messages = await PromptForApprovalsAsync(ux, this._approvalRequests);
this._approvalRequests.Clear();
return messages;
return Task.FromResult<IList<FollowUpAction>?>(actions);
}
private static async Task<List<ChatMessage>?> PromptForApprovalsAsync(HarnessUXContainer ux, List<ToolApprovalRequestContent> approvalRequests)
private ChoiceFollowUpQuestion BuildApprovalQuestion(ToolApprovalRequestContent request)
{
if (approvalRequests.Count == 0)
string toolName = request.ToolCall is FunctionCallContent fc
? ToolCallFormatter.Format(this._formatters, fc)
: request.ToolCall?.ToString() ?? "unknown";
var choices = new List<string>
{
return null;
}
"Approve this call",
"Always approve this tool (any arguments)",
"Always approve this tool with these arguments",
"Deny",
};
var responses = new List<AIContent>();
foreach (var request in approvalRequests)
{
string toolName = request.ToolCall is FunctionCallContent fc
? ToolCallFormatter.Format(fc)
: request.ToolCall?.ToString() ?? "unknown";
string prompt = $"🔐 Tool approval: {toolName}";
var choices = new List<string>
return new ChoiceFollowUpQuestion(
Prompt: prompt,
Choices: choices,
AllowCustomText: false,
Continuation: async (selection, ux) =>
{
"Approve this call",
"Always approve this tool (any arguments)",
"Always approve this tool with these arguments",
"Deny",
};
AIContent response = selection switch
{
"Always approve this tool (any arguments)" => request.CreateAlwaysApproveToolResponse("User chose to always approve this tool"),
"Always approve this tool with these arguments" => request.CreateAlwaysApproveToolWithArgumentsResponse("User chose to always approve this tool with these arguments"),
"Deny" => request.CreateResponse(approved: false, reason: "User denied"),
_ => request.CreateResponse(approved: true, reason: "User approved"),
};
string selection = await ux.ReadSelectionAsync($"🔐 Tool approval: {toolName}", choices);
AIContent response = selection switch
{
"Always approve this tool (any arguments)" => request.CreateAlwaysApproveToolResponse("User chose to always approve this tool"),
"Always approve this tool with these arguments" => request.CreateAlwaysApproveToolWithArgumentsResponse("User chose to always approve this tool with these arguments"),
"Deny" => request.CreateResponse(approved: false, reason: "User denied"),
_ => request.CreateResponse(approved: true, reason: "User approved"),
};
string action = selection switch
{
"Always approve this tool (any arguments)" => "✅ Always approved (any args)",
"Always approve this tool with these arguments" => "✅ Always approved (these args)",
"Deny" => "❌ Denied",
_ => "✅ Approved",
};
string action = selection switch
{
"Always approve this tool (any arguments)" => "✅ Always approved (any args)",
"Always approve this tool with these arguments" => "✅ Always approved (these args)",
"Deny" => "❌ Denied",
_ => "✅ Approved",
};
await ux.WriteInfoLineAsync($" {action}", ConsoleColor.DarkGray);
ConsoleColor answerColor = selection == "Deny" ? ConsoleColor.Red : ConsoleColor.Green;
string formatted = $"🔹 {prompt}\n └─ {AnsiEscapes.SetForegroundColor(answerColor)}{action}{AnsiEscapes.ResetAttributes}";
await ux.WriteInfoLineAsync(formatted, ConsoleColor.Gray).ConfigureAwait(false);
responses.Add(response);
}
return [new ChatMessage(ChatRole.User, responses)];
return new ChatMessage(ChatRole.User, [response]);
});
}
}
@@ -1,5 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
using Harness.Shared.Console.ToolFormatters;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.Observers;
@@ -8,14 +10,26 @@ namespace Harness.Shared.Console.Observers;
/// Displays tool call notifications (🔧) for <see cref="FunctionCallContent"/>
/// and <see cref="ToolCallContent"/> items in the response stream.
/// </summary>
internal sealed class ToolCallDisplayObserver : ConsoleObserver
public sealed class ToolCallDisplayObserver : ConsoleObserver
{
private readonly IReadOnlyList<ToolCallFormatter> _formatters;
/// <summary>
/// Initializes a new instance of the <see cref="ToolCallDisplayObserver"/> class.
/// </summary>
/// <param name="formatters">Optional list of tool formatters. When <see langword="null"/>,
/// the default formatters from <see cref="ToolCallFormatter.BuildDefaultToolFormatters"/> are used.</param>
public ToolCallDisplayObserver(IReadOnlyList<ToolCallFormatter>? formatters = null)
{
this._formatters = formatters ?? ToolCallFormatter.BuildDefaultToolFormatters();
}
/// <inheritdoc/>
public override async Task OnContentAsync(HarnessUXContainer ux, AIContent content)
public override async Task OnContentAsync(IUXStateDriver ux, AIContent content, AIAgent agent, AgentSession session)
{
if (content is FunctionCallContent functionCall)
{
await ux.WriteInfoLineAsync($"🔧 Calling tool: {ToolCallFormatter.Format(functionCall)}...", ConsoleColor.DarkYellow);
await ux.WriteInfoLineAsync($"🔧 Calling tool: {ToolCallFormatter.Format(this._formatters, functionCall)}...", ConsoleColor.DarkYellow);
}
else if (content is ToolCallContent toolCall)
{
@@ -1,288 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Text;
using System.Text.Json;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.Observers;
/// <summary>
/// Formats <see cref="FunctionCallContent"/> instances into human-readable strings
/// for console display.
/// </summary>
public static class ToolCallFormatter
{
/// <summary>
/// Returns a formatted string for the given tool call, with human-readable
/// details for known tools (todos, mode, sub-agents, web tools).
/// </summary>
/// <param name="call">The function call content to format.</param>
/// <returns>A formatted string describing the tool call.</returns>
public static string Format(FunctionCallContent call)
{
string? detail = call.Name switch
{
// Todo tools
"TodoList_Add" => FormatAddTodos(call),
"TodoList_Complete" => FormatIdList(call, "ids", "Complete"),
"TodoList_Remove" => FormatIdList(call, "ids", "Remove"),
"TodoList_GetRemaining" => null,
"TodoList_GetAll" => null,
// Mode tools
"AgentMode_Set" => FormatStringArg(call, "mode"),
"AgentMode_Get" => null,
// Sub-agent tools
"SubAgents_StartTask" => FormatStartSubTask(call),
"SubAgents_WaitForFirstCompletion" => FormatIdList(call, "taskIds", "Wait for"),
"SubAgents_GetTaskResults" => FormatSingleId(call, "taskId"),
"SubAgents_GetAllTasks" => null,
"SubAgents_ContinueTask" => FormatContinueTask(call),
"SubAgents_ClearCompletedTask" => FormatSingleId(call, "taskId"),
// File memory tools
"FileMemory_SaveFile" => FormatSaveFile(call),
"FileMemory_ReadFile" => FormatStringArg(call, "fileName"),
"FileMemory_DeleteFile" => FormatStringArg(call, "fileName"),
"FileMemory_ListFiles" => null,
"FileMemory_SearchFiles" => FormatSearchFiles(call),
// External tools
"web_search" => FormatStringArg(call, "query"),
"DownloadUri" => FormatStringArg(call, "uri"),
_ => FormatFallback(call),
};
return detail is not null ? $"{call.Name} {detail}" : call.Name;
}
private static string? FormatAddTodos(FunctionCallContent call)
{
if (call.Arguments?.TryGetValue("todos", out object? todosObj) != true || todosObj is null)
{
return null;
}
var titles = new List<string>();
if (todosObj is JsonElement jsonArray && jsonArray.ValueKind == JsonValueKind.Array)
{
foreach (JsonElement item in jsonArray.EnumerateArray())
{
string? title = item.TryGetProperty("title", out JsonElement titleElement)
? titleElement.GetString()
: null;
if (!string.IsNullOrEmpty(title))
{
titles.Add(title);
}
}
}
if (titles.Count == 0)
{
return null;
}
var sb = new StringBuilder();
sb.Append($"({titles.Count} item{(titles.Count == 1 ? "" : "s")})");
foreach (string title in titles)
{
sb.Append($"\n • {title}");
}
return sb.ToString();
}
private static string? FormatIdList(FunctionCallContent call, string paramName, string verb)
{
List<int>? ids = GetIntList(call, paramName);
if (ids is null || ids.Count == 0)
{
return null;
}
return $"({verb} #{string.Join(", #", ids)})";
}
private static string? FormatSingleId(FunctionCallContent call, string paramName)
{
int? id = GetInt(call, paramName);
return id.HasValue ? $"(task #{id.Value})" : null;
}
private static string? FormatStartSubTask(FunctionCallContent call)
{
string? agentName = GetString(call, "agentName");
string? description = GetString(call, "description");
if (agentName is null && description is null)
{
return null;
}
var sb = new StringBuilder("(");
if (agentName is not null)
{
sb.Append($"agent: {agentName}");
}
if (description is not null)
{
if (agentName is not null)
{
sb.Append(", ");
}
sb.Append($"\"{Truncate(description, 60)}\"");
}
sb.Append(')');
return sb.ToString();
}
private static string? FormatContinueTask(FunctionCallContent call)
{
int? taskId = GetInt(call, "taskId");
string? text = GetString(call, "text");
if (!taskId.HasValue)
{
return null;
}
return text is not null
? $"(task #{taskId.Value}, \"{Truncate(text, 50)}\")"
: $"(task #{taskId.Value})";
}
private static string? FormatSaveFile(FunctionCallContent call)
{
string? fileName = GetString(call, "fileName");
string? description = GetString(call, "description");
if (fileName is null)
{
return null;
}
return string.IsNullOrEmpty(description)
? $"({fileName})"
: $"({fileName}, with description)";
}
private static string? FormatSearchFiles(FunctionCallContent call)
{
string? pattern = GetString(call, "regexPattern");
string? filePattern = GetString(call, "filePattern");
if (pattern is null)
{
return null;
}
return string.IsNullOrEmpty(filePattern)
? $"(/{pattern}/)"
: $"(/{pattern}/ in {filePattern})";
}
private static string? FormatStringArg(FunctionCallContent call, string paramName)
{
string? value = GetString(call, paramName);
return value is not null ? $"({value})" : null;
}
private static string? FormatFallback(FunctionCallContent call)
{
if (call.Arguments is null || call.Arguments.Count == 0)
{
return null;
}
var parts = new List<string>();
foreach (var kvp in call.Arguments)
{
string? stringValue = kvp.Value switch
{
JsonElement je => je.ValueKind switch
{
JsonValueKind.String => je.GetString(),
JsonValueKind.Number => je.GetRawText(),
JsonValueKind.True => "true",
JsonValueKind.False => "false",
_ => null,
},
not null => kvp.Value.ToString(),
_ => null,
};
if (stringValue is not null)
{
parts.Add($"{kvp.Key}: {Truncate(stringValue, 40)}");
}
}
return parts.Count > 0 ? $"({string.Join(", ", parts)})" : null;
}
private static string? GetString(FunctionCallContent call, string paramName)
{
if (call.Arguments?.TryGetValue(paramName, out object? value) != true || value is null)
{
return null;
}
return value switch
{
JsonElement je when je.ValueKind == JsonValueKind.String => je.GetString(),
string s => s,
_ => value.ToString(),
};
}
private static int? GetInt(FunctionCallContent call, string paramName)
{
if (call.Arguments?.TryGetValue(paramName, out object? value) != true || value is null)
{
return null;
}
return value switch
{
JsonElement je when je.ValueKind == JsonValueKind.Number => je.GetInt32(),
int i => i,
_ => int.TryParse(value.ToString(), out int parsed) ? parsed : null,
};
}
private static List<int>? GetIntList(FunctionCallContent call, string paramName)
{
if (call.Arguments?.TryGetValue(paramName, out object? value) != true || value is null)
{
return null;
}
var result = new List<int>();
if (value is JsonElement je && je.ValueKind == JsonValueKind.Array)
{
foreach (JsonElement item in je.EnumerateArray())
{
if (item.ValueKind == JsonValueKind.Number)
{
result.Add(item.GetInt32());
}
}
}
return result.Count > 0 ? result : null;
}
private static string Truncate(string text, int maxLength)
{
return text.Length <= maxLength ? text : string.Concat(text.AsSpan(0, maxLength), "…");
}
}
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.Observers;
@@ -7,7 +8,7 @@ namespace Harness.Shared.Console.Observers;
/// <summary>
/// Displays token usage statistics (📊) from the response stream.
/// </summary>
internal sealed class UsageDisplayObserver : ConsoleObserver
public sealed class UsageDisplayObserver : ConsoleObserver
{
private readonly int? _maxContextWindowTokens;
private readonly int? _maxOutputTokens;
@@ -24,7 +25,7 @@ internal sealed class UsageDisplayObserver : ConsoleObserver
}
/// <inheritdoc/>
public override Task OnContentAsync(HarnessUXContainer ux, AIContent content)
public override Task OnContentAsync(IUXStateDriver ux, AIContent content, AIAgent agent, AgentSession session)
{
if (content is UsageContent usage)
{
@@ -5,7 +5,7 @@ namespace Harness.Shared.Console;
/// <summary>
/// Represents the type of an output entry in the console conversation.
/// </summary>
public enum OutputEntryType
internal enum OutputEntryType
{
/// <summary>User input echo (e.g. "You: hello").</summary>
UserInput,
@@ -25,9 +25,10 @@ public enum OutputEntryType
/// <summary>
/// Represents a single output entry in the console conversation history.
/// These entries are rendered by the <see cref="HarnessAppComponent"/> via its render delegate.
/// Used internally by <see cref="HarnessConsoleUXStateDriver"/> to track
/// the in-progress streaming entry and last-entry type for spacing decisions.
/// </summary>
/// <param name="Type">The type of output entry.</param>
/// <param name="Text">The text content of the entry.</param>
/// <param name="Color">Optional foreground color for rendering.</param>
public record OutputEntry(OutputEntryType Type, string Text, ConsoleColor? Color = null);
internal sealed record OutputEntry(OutputEntryType Type, string Text, ConsoleColor? Color = null);
@@ -0,0 +1,51 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Text.Json;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Catch-all formatter that handles any tool not matched by a more specific formatter.
/// Displays a generic summary of the tool's arguments. This formatter should always be
/// placed last in the formatter list.
/// </summary>
public sealed class FallbackToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) => true;
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call)
{
if (call.Arguments is null || call.Arguments.Count == 0)
{
return null;
}
var parts = new List<string>();
foreach (var kvp in call.Arguments)
{
string? stringValue = kvp.Value switch
{
JsonElement je => je.ValueKind switch
{
JsonValueKind.String => je.GetString(),
JsonValueKind.Number => je.GetRawText(),
JsonValueKind.True => "true",
JsonValueKind.False => "false",
_ => null,
},
not null => kvp.Value.ToString(),
_ => null,
};
if (stringValue is not null)
{
parts.Add($"{kvp.Key}: {Truncate(stringValue, 40)}");
}
}
return parts.Count > 0 ? $"({string.Join(", ", parts)})" : null;
}
}
@@ -0,0 +1,61 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Formats <c>FileMemory_*</c> tool calls, showing file names and search patterns
/// with tree-view corners for save operations.
/// </summary>
public sealed class FileMemoryToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("FileMemory_", StringComparison.Ordinal);
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call) => call.Name switch
{
"FileMemory_SaveFile" => FormatSaveFile(call),
"FileMemory_ReadFile" => FormatStringArg(call, "fileName"),
"FileMemory_DeleteFile" => FormatStringArg(call, "fileName"),
"FileMemory_SearchFiles" => FormatSearchFiles(call),
_ => null,
};
private static string? FormatSaveFile(FunctionCallContent call)
{
string? fileName = GetStringArgumentValue(call, "fileName");
string? description = GetStringArgumentValue(call, "description");
if (fileName is null)
{
return null;
}
return string.IsNullOrEmpty(description)
? $"\n └─ {fileName}"
: $"\n └─ {fileName} (with description)";
}
private static string? FormatSearchFiles(FunctionCallContent call)
{
string? pattern = GetStringArgumentValue(call, "regexPattern");
string? filePattern = GetStringArgumentValue(call, "filePattern");
if (pattern is null)
{
return null;
}
return string.IsNullOrEmpty(filePattern)
? $"(/{pattern}/)"
: $"(/{pattern}/ in {filePattern})";
}
private static string? FormatStringArg(FunctionCallContent call, string paramName)
{
string? value = GetStringArgumentValue(call, paramName);
return value is not null ? $"({value})" : null;
}
}
@@ -0,0 +1,27 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Formats <c>AgentMode_*</c> tool calls, showing the target mode for Set operations.
/// </summary>
public sealed class ModeToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("AgentMode_", StringComparison.Ordinal);
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call) => call.Name switch
{
"AgentMode_Set" => FormatStringArg(call, "mode"),
_ => null,
};
private static string? FormatStringArg(FunctionCallContent call, string paramName)
{
string? value = GetStringArgumentValue(call, paramName);
return value is not null ? $"({value})" : null;
}
}
@@ -0,0 +1,101 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Text;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Formats <c>SubAgents_*</c> tool calls with human-readable details
/// for task start, continue, wait, and result retrieval operations.
/// </summary>
public sealed class SubAgentToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("SubAgents_", StringComparison.Ordinal);
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call) => call.Name switch
{
"SubAgents_StartTask" => FormatStartSubTask(call),
"SubAgents_WaitForFirstCompletion" => FormatIdList(call, "taskIds", "Wait for"),
"SubAgents_GetTaskResults" => FormatSingleId(call, "taskId"),
"SubAgents_ContinueTask" => FormatContinueTask(call),
"SubAgents_ClearCompletedTask" => FormatSingleId(call, "taskId"),
_ => null,
};
private static string? FormatStartSubTask(FunctionCallContent call)
{
string? agentName = GetStringArgumentValue(call, "agentName");
string? description = GetStringArgumentValue(call, "description");
if (agentName is null && description is null)
{
return null;
}
var sb = new StringBuilder();
if (agentName is not null && description is not null)
{
sb.Append($"\n ├─ Agent: {agentName}");
sb.Append($"\n └─ \"{Truncate(description, 80)}\"");
}
else if (agentName is not null)
{
sb.Append($"\n └─ Agent: {agentName}");
}
else
{
sb.Append($"\n └─ \"{Truncate(description!, 80)}\"");
}
return sb.ToString();
}
private static string? FormatIdList(FunctionCallContent call, string paramName, string verb)
{
List<int>? ids = GetIntListArgumentValue(call, paramName);
if (ids is null || ids.Count == 0)
{
return null;
}
var sb = new StringBuilder();
for (int i = 0; i < ids.Count; i++)
{
string connector = i < ids.Count - 1 ? "├─" : "└─";
sb.Append($"\n {connector} {verb} #{ids[i]}");
}
return sb.ToString();
}
private static string? FormatSingleId(FunctionCallContent call, string paramName)
{
int? id = GetIntArgumentValue(call, paramName);
return id.HasValue ? $"(task #{id.Value})" : null;
}
private static string? FormatContinueTask(FunctionCallContent call)
{
int? taskId = GetIntArgumentValue(call, "taskId");
string? text = GetStringArgumentValue(call, "text");
if (!taskId.HasValue)
{
return null;
}
if (text is not null)
{
var sb = new StringBuilder();
sb.Append($"\n ├─ Task #{taskId.Value}");
sb.Append($"\n └─ \"{Truncate(text, 80)}\"");
return sb.ToString();
}
return $"\n └─ Task #{taskId.Value}";
}
}
@@ -0,0 +1,84 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Text;
using System.Text.Json;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Formats <c>TodoList_*</c> tool calls with tree-view output for added items
/// and structured output for complete/remove operations.
/// </summary>
public sealed class TodoToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("TodoList_", StringComparison.Ordinal);
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call) => call.Name switch
{
"TodoList_Add" => FormatAddTodos(call),
"TodoList_Complete" => FormatIdList(call, "ids", "Complete"),
"TodoList_Remove" => FormatIdList(call, "ids", "Remove"),
_ => null,
};
private static string? FormatAddTodos(FunctionCallContent call)
{
if (call.Arguments?.TryGetValue("todos", out object? todosObj) != true || todosObj is null)
{
return null;
}
var titles = new List<string>();
if (todosObj is JsonElement jsonArray && jsonArray.ValueKind == JsonValueKind.Array)
{
foreach (JsonElement item in jsonArray.EnumerateArray())
{
string? title = item.TryGetProperty("title", out JsonElement titleElement)
? titleElement.GetString()
: null;
if (!string.IsNullOrEmpty(title))
{
titles.Add(title);
}
}
}
if (titles.Count == 0)
{
return null;
}
var sb = new StringBuilder();
sb.Append($"({titles.Count} item{(titles.Count == 1 ? "" : "s")})");
for (int i = 0; i < titles.Count; i++)
{
string connector = i < titles.Count - 1 ? "├─" : "└─";
sb.Append($"\n {connector} {titles[i]}");
}
return sb.ToString();
}
private static string? FormatIdList(FunctionCallContent call, string paramName, string verb)
{
List<int>? ids = GetIntListArgumentValue(call, paramName);
if (ids is null || ids.Count == 0)
{
return null;
}
var sb = new StringBuilder();
for (int i = 0; i < ids.Count; i++)
{
string connector = i < ids.Count - 1 ? "├─" : "└─";
sb.Append($"\n {connector} {verb} #{ids[i]}");
}
return sb.ToString();
}
}
@@ -0,0 +1,135 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Text.Json;
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Base class for tool call formatters that produce human-readable display strings
/// for <see cref="FunctionCallContent"/> items shown in the console.
/// </summary>
public abstract class ToolCallFormatter
{
/// <summary>
/// Returns <see langword="true"/> if this formatter can handle the given function call.
/// </summary>
/// <param name="call">The function call content to check.</param>
/// <returns><see langword="true"/> if this formatter should be used; otherwise <see langword="false"/>.</returns>
public abstract bool CanFormat(FunctionCallContent call);
/// <summary>
/// Returns the detail portion of the formatted output for the given tool call,
/// or <see langword="null"/> if only the tool name should be displayed.
/// </summary>
/// <param name="call">The function call content to format.</param>
/// <returns>A detail string to append after the tool name, or <see langword="null"/>.</returns>
public abstract string? FormatDetail(FunctionCallContent call);
/// <summary>
/// Formats a tool call using the first matching formatter from the provided list.
/// Returns <c>"{toolName} {detail}"</c> when a formatter produces detail,
/// or just <c>"{toolName}"</c> otherwise.
/// </summary>
internal static string Format(IReadOnlyList<ToolCallFormatter> formatters, FunctionCallContent call)
{
foreach (var formatter in formatters)
{
if (formatter.CanFormat(call))
{
string? detail = formatter.FormatDetail(call);
return detail is not null ? $"{call.Name} {detail}" : call.Name;
}
}
return call.Name;
}
/// <summary>
/// Creates the default list of tool call formatters. The <see cref="FallbackToolFormatter"/>
/// is always last. Users can call this method and combine the result with their own formatters.
/// </summary>
/// <returns>A list of all built-in tool call formatters.</returns>
public static List<ToolCallFormatter> BuildDefaultToolFormatters()
{
return
[
new TodoToolFormatter(),
new ModeToolFormatter(),
new SubAgentToolFormatter(),
new FileMemoryToolFormatter(),
new WebSearchToolFormatter(),
new FallbackToolFormatter(),
];
}
/// <summary>
/// Extracts a string argument value from a function call.
/// </summary>
protected static string? GetStringArgumentValue(FunctionCallContent call, string paramName)
{
if (call.Arguments?.TryGetValue(paramName, out object? value) != true || value is null)
{
return null;
}
return value switch
{
JsonElement je when je.ValueKind == JsonValueKind.String => je.GetString(),
string s => s,
_ => value.ToString(),
};
}
/// <summary>
/// Extracts an integer argument value from a function call.
/// </summary>
protected static int? GetIntArgumentValue(FunctionCallContent call, string paramName)
{
if (call.Arguments?.TryGetValue(paramName, out object? value) != true || value is null)
{
return null;
}
return value switch
{
JsonElement je when je.ValueKind == JsonValueKind.Number => je.GetInt32(),
int i => i,
_ => int.TryParse(value.ToString(), out int parsed) ? parsed : null,
};
}
/// <summary>
/// Extracts a list of integer argument values from a function call.
/// </summary>
protected static List<int>? GetIntListArgumentValue(FunctionCallContent call, string paramName)
{
if (call.Arguments?.TryGetValue(paramName, out object? value) != true || value is null)
{
return null;
}
var result = new List<int>();
if (value is JsonElement je && je.ValueKind == JsonValueKind.Array)
{
foreach (JsonElement item in je.EnumerateArray())
{
if (item.ValueKind == JsonValueKind.Number)
{
result.Add(item.GetInt32());
}
}
}
return result.Count > 0 ? result : null;
}
/// <summary>
/// Truncates a string to the specified maximum length, appending an ellipsis if truncated.
/// </summary>
protected static string Truncate(string text, int maxLength)
{
return text.Length <= maxLength ? text : string.Concat(text.AsSpan(0, maxLength), "…");
}
}
@@ -0,0 +1,22 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Formats <c>web_search</c> tool calls, showing the search query.
/// </summary>
public sealed class WebSearchToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) =>
call.Name is "web_search";
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call)
{
string? value = GetStringArgumentValue(call, "query");
return value is not null ? $"({value})" : null;
}
}
@@ -0,0 +1,23 @@
// Copyright (c) Microsoft. All rights reserved.
using Harness.Shared.Console.ToolFormatters;
using Microsoft.Extensions.AI;
namespace SampleApp;
/// <summary>
/// Formats <c>DownloadUri</c> tool calls, showing the target URI.
/// </summary>
public sealed class DownloadUriToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) =>
call.Name is "DownloadUri";
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call)
{
string? value = GetStringArgumentValue(call, "uri");
return value is not null ? $"({value})" : null;
}
}
@@ -8,7 +8,8 @@
//
// Special commands:
// /todos — Display the current todo list without invoking the agent.
// exit — End the session.
// /mode — Get or set the current agent mode.
// /exit — End the session.
#pragma warning disable OPENAI001 // Suppress experimental API warnings for Responses API usage.
#pragma warning disable MAAI001 // Suppress experimental API warnings for Agents AI experiments.
@@ -16,6 +17,7 @@
using System.ClientModel.Primitives;
using Azure.Identity;
using Harness.Shared.Console;
using Harness.Shared.Console.ToolFormatters;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
using OpenAI;
@@ -158,13 +160,15 @@ AIAgent agent =
// Run the interactive console session using the shared HarnessConsole helper.
await HarnessConsole.RunAgentAsync(
agent,
title: "Research Assistant",
userPrompt: "Enter a research topic to get started.",
new HarnessConsoleOptions
{
MaxContextWindowTokens = MaxContextWindowTokens,
MaxOutputTokens = MaxOutputTokens,
EnablePlanningUx = true,
PlanningModeName = "plan",
ExecutionModeName = "execute"
Observers = HarnessConsoleOptions.BuildObserversWithPlanning(
agent,
planModeName: "plan",
executionModeName: "execute",
maxContextWindowTokens: MaxContextWindowTokens,
maxOutputTokens: MaxOutputTokens,
toolFormatters: [new DownloadUriToolFormatter(), .. ToolCallFormatter.BuildDefaultToolFormatters()]),
CommandHandlers = HarnessConsoleOptions.BuildDefaultCommandHandlers(agent),
});
@@ -6,7 +6,7 @@
// equipped with Foundry's hosted web search tool.
//
// Special commands:
// exit — End the session.
// /exit — End the session.
#pragma warning disable OPENAI001 // Suppress experimental API warnings for Responses API usage.
#pragma warning disable MAAI001 // Suppress experimental API warnings for Agents AI experiments.
@@ -103,5 +103,4 @@ AIAgent parentAgent =
// Run the interactive console session.
await HarnessConsole.RunAgentAsync(
parentAgent,
title: "Stock Price Researcher (SubAgents Demo)",
userPrompt: "Enter a list of stock tickers (e.g., BAC, MSFT, BA):");
@@ -8,7 +8,7 @@
// Ask the agent to analyze the data, produce summaries, or create new output files.
//
// Special commands:
// exit — End the session.
// /exit — End the session.
#pragma warning disable OPENAI001 // Suppress experimental API warnings for Responses API usage.
#pragma warning disable MAAI001 // Suppress experimental API warnings for Agents AI experiments.
@@ -85,5 +85,4 @@ AIAgent agent =
// Run the interactive console session.
await HarnessConsole.RunAgentAsync(
agent,
title: "Data Processing Assistant",
userPrompt: "Ask me to analyze the data files, produce summaries, or create output files.");
@@ -17,6 +17,7 @@ namespace Microsoft.Agents.AI;
/// <see cref="HarnessAgent"/> assembles the following pipeline from a caller-supplied <see cref="IChatClient"/>:
/// <list type="number">
/// <item><description><see cref="FunctionInvokingChatClient"/> — automatic function/tool invocation.</description></item>
/// <item><description><see cref="MessageInjectingChatClient"/> — allows external code to inject messages into the conversation mid-stream.</description></item>
/// <item><description><see cref="PerServiceCallChatHistoryPersistingChatClient"/> — persists chat history after every individual service call within a function-invocation loop.</description></item>
/// <item><description><see cref="AIContextProviderChatClient"/> with a <see cref="CompactionProvider"/> — applies context-window compaction before each call so long function-invocation loops do not overflow the context window.</description></item>
/// </list>
@@ -110,6 +111,7 @@ public sealed class HarnessAgent : DelegatingAIAgent
return chatClient
.AsBuilder()
.UseFunctionInvocation()
.UseMessageInjection()
.UsePerServiceCallChatHistoryPersistence()
.UseAIContextProviders(compactionProvider)
.BuildAIAgent(new ChatClientAgentOptions
@@ -0,0 +1,546 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.Agents.AI.Workflows.Execution;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
public sealed class RouteBuilderTests
{
public enum HandlerOverload
{
SyncWithCancellation = 0,
SyncWithoutCancellation = 1,
AsyncWithCancellation = 2,
AsyncWithoutCancellation = 3,
}
private sealed record TestPayload(string Value);
private sealed class HandlerInvocation
{
public object? Message { get; private set; }
public IWorkflowContext? Context { get; private set; }
public CancellationToken CancellationToken { get; private set; }
public int InvocationCount { get; private set; }
public void Capture(object? message, IWorkflowContext context, CancellationToken cancellationToken = default)
{
this.Message = message;
this.Context = context;
this.CancellationToken = cancellationToken;
this.InvocationCount++;
}
}
private sealed class TestExternalRequestContext : IExternalRequestContext, IExternalRequestSink
{
public List<RequestPort> RegisteredPorts { get; } = [];
public List<ExternalRequest> PostedRequests { get; } = [];
public IExternalRequestSink RegisterPort(RequestPort port)
{
this.RegisteredPorts.Add(port);
return this;
}
public ValueTask PostAsync(ExternalRequest request)
{
this.PostedRequests.Add(request);
return default;
}
}
[Theory]
[InlineData(HandlerOverload.SyncWithCancellation)]
[InlineData(HandlerOverload.SyncWithoutCancellation)]
[InlineData(HandlerOverload.AsyncWithCancellation)]
[InlineData(HandlerOverload.AsyncWithoutCancellation)]
public async Task AddHandler_VoidOverloads_RouteExpectedMessageAsync(HandlerOverload overload)
{
// Arrange
RouteBuilder routeBuilder = new(null);
HandlerInvocation invocation = new();
CancellationToken cancellationToken = new CancellationTokenSource().Token;
RegisterVoidHandler(routeBuilder, invocation, overload);
MessageRouter router = routeBuilder.Build();
TestWorkflowContext context = new("executor");
// Act
CallResult? result = await router.RouteMessageAsync("hello", context, cancellationToken: cancellationToken);
// Assert
result.Should().NotBeNull();
result!.IsSuccess.Should().BeTrue();
result.IsVoid.Should().BeTrue();
result.Result.Should().BeNull();
invocation.InvocationCount.Should().Be(1);
invocation.Message.Should().Be("hello");
invocation.Context.Should().BeSameAs(context);
if (UsesCancellationToken(overload))
{
invocation.CancellationToken.Should().Be(cancellationToken);
}
}
[Theory]
[InlineData(HandlerOverload.SyncWithCancellation)]
[InlineData(HandlerOverload.SyncWithoutCancellation)]
[InlineData(HandlerOverload.AsyncWithCancellation)]
[InlineData(HandlerOverload.AsyncWithoutCancellation)]
public async Task AddHandler_ResultOverloads_RouteExpectedMessageAsync(HandlerOverload overload)
{
// Arrange
RouteBuilder routeBuilder = new(null);
HandlerInvocation invocation = new();
CancellationToken cancellationToken = new CancellationTokenSource().Token;
RegisterResultHandler(routeBuilder, invocation, overload);
MessageRouter router = routeBuilder.Build();
TestWorkflowContext context = new("executor");
// Act
CallResult? result = await router.RouteMessageAsync("hello", context, cancellationToken: cancellationToken);
// Assert
result.Should().NotBeNull();
result!.IsSuccess.Should().BeTrue();
result.IsVoid.Should().BeFalse();
result.Result.Should().Be("HELLO");
router.DefaultOutputTypes.Should().Contain(typeof(string));
invocation.InvocationCount.Should().Be(1);
invocation.Message.Should().Be("hello");
invocation.Context.Should().BeSameAs(context);
if (UsesCancellationToken(overload))
{
invocation.CancellationToken.Should().Be(cancellationToken);
}
}
[Theory]
[InlineData(HandlerOverload.SyncWithCancellation)]
[InlineData(HandlerOverload.SyncWithoutCancellation)]
[InlineData(HandlerOverload.AsyncWithCancellation)]
[InlineData(HandlerOverload.AsyncWithoutCancellation)]
public async Task AddCatchAll_VoidOverloads_RouteUnexpectedMessageAsync(HandlerOverload overload)
{
// Arrange
RouteBuilder routeBuilder = new(null);
HandlerInvocation invocation = new();
CancellationToken cancellationToken = new CancellationTokenSource().Token;
TestPayload payload = new("hello");
RegisterVoidCatchAll(routeBuilder, invocation, overload);
MessageRouter router = routeBuilder.Build();
TestWorkflowContext context = new("executor");
// Act
CallResult? result = await router.RouteMessageAsync(payload, context, cancellationToken: cancellationToken);
// Assert
result.Should().NotBeNull();
result!.IsSuccess.Should().BeTrue();
result.IsVoid.Should().BeTrue();
result.Result.Should().BeNull();
invocation.InvocationCount.Should().Be(1);
invocation.Message.Should().BeEquivalentTo(new PortableValue(payload));
invocation.Context.Should().BeSameAs(context);
if (UsesCancellationToken(overload))
{
invocation.CancellationToken.Should().Be(cancellationToken);
}
}
[Theory]
[InlineData(HandlerOverload.SyncWithCancellation)]
[InlineData(HandlerOverload.SyncWithoutCancellation)]
[InlineData(HandlerOverload.AsyncWithCancellation)]
[InlineData(HandlerOverload.AsyncWithoutCancellation)]
public async Task AddCatchAll_ResultOverloads_RouteUnexpectedMessageAsync(HandlerOverload overload)
{
// Arrange
RouteBuilder routeBuilder = new(null);
HandlerInvocation invocation = new();
CancellationToken cancellationToken = new CancellationTokenSource().Token;
TestPayload payload = new("hello");
RegisterResultCatchAll(routeBuilder, invocation, overload);
MessageRouter router = routeBuilder.Build();
TestWorkflowContext context = new("executor");
// Act
CallResult? result = await router.RouteMessageAsync(payload, context, cancellationToken: cancellationToken);
// Assert
result.Should().NotBeNull();
result!.IsSuccess.Should().BeTrue();
result.IsVoid.Should().BeFalse();
result.Result.Should().Be("HELLO");
invocation.InvocationCount.Should().Be(1);
invocation.Message.Should().BeEquivalentTo(new PortableValue(payload));
invocation.Context.Should().BeSameAs(context);
if (UsesCancellationToken(overload))
{
invocation.CancellationToken.Should().Be(cancellationToken);
}
}
[Fact]
public async Task AddHandlerUntyped_VoidAndResultOverloads_RouteExpectedMessageAsync()
{
// Arrange
RouteBuilder routeBuilder = new(null);
HandlerInvocation voidInvocation = new();
HandlerInvocation resultInvocation = new();
CancellationToken cancellationToken = new CancellationTokenSource().Token;
routeBuilder.AddHandlerUntyped(typeof(string), (message, context, token) =>
{
voidInvocation.Capture(message, context, token);
return default;
});
routeBuilder.AddHandlerUntyped<int>(typeof(int), (message, context, token) =>
{
resultInvocation.Capture(message, context, token);
return new((int)message + 1);
});
MessageRouter router = routeBuilder.Build();
TestWorkflowContext context = new("executor");
// Act
CallResult? voidResult = await router.RouteMessageAsync("hello", context, cancellationToken: cancellationToken);
CallResult? typedResult = await router.RouteMessageAsync(41, context, cancellationToken: cancellationToken);
// Assert
voidResult.Should().NotBeNull();
voidResult!.IsVoid.Should().BeTrue();
voidInvocation.Message.Should().Be("hello");
voidInvocation.Context.Should().BeSameAs(context);
voidInvocation.CancellationToken.Should().Be(cancellationToken);
typedResult.Should().NotBeNull();
typedResult!.Result.Should().Be(42);
router.DefaultOutputTypes.Should().Contain(typeof(int));
resultInvocation.Message.Should().Be(41);
resultInvocation.Context.Should().BeSameAs(context);
resultInvocation.CancellationToken.Should().Be(cancellationToken);
}
[Fact]
public void AddHandler_ForPortableValue_ThrowsInvalidOperationException()
{
// Arrange
RouteBuilder routeBuilder = new(null);
// Act
Action act = () => routeBuilder.AddHandler<PortableValue>((message, context) => { });
// Assert
act.Should().Throw<InvalidOperationException>()
.WithMessage("*Use AddCatchAll()*");
}
[Fact]
public void AddHandler_DuplicateRegistrationWithoutOverwrite_ThrowsArgumentException()
{
// Arrange
RouteBuilder routeBuilder = new(null);
routeBuilder.AddHandler<string>((message, context) => { });
// Act
Action act = () => routeBuilder.AddHandler<string>((message, context) => { });
// Assert
act.Should().Throw<ArgumentException>()
.WithMessage("*already registered*");
}
[Fact]
public void AddHandler_OverwriteWithoutExistingRegistration_ThrowsArgumentException()
{
// Arrange
RouteBuilder routeBuilder = new(null);
// Act
Action act = () => routeBuilder.AddHandler<string>((message, context) => { }, overwrite: true);
// Assert
act.Should().Throw<ArgumentException>()
.WithMessage("*has not yet been registered*");
}
[Fact]
public async Task AddHandler_OverwriteExistingRegistration_RoutesUpdatedHandlerAsync()
{
// Arrange
RouteBuilder routeBuilder = new(null);
routeBuilder.AddHandler<string>((message, context) => context.SendMessageAsync("first"));
routeBuilder.AddHandler<string>((message, context) => context.SendMessageAsync("second"), overwrite: true);
MessageRouter router = routeBuilder.Build();
TestWorkflowContext context = new("executor");
// Act
_ = await router.RouteMessageAsync("hello", context);
// Assert
context.SentMessages.Should().ContainSingle().Which.Should().Be("second");
}
[Fact]
public void AddCatchAll_DuplicateRegistrationWithoutOverwrite_ThrowsInvalidOperationException()
{
// Arrange
RouteBuilder routeBuilder = new(null);
routeBuilder.AddCatchAll((message, context) => { });
// Act
Action act = () => routeBuilder.AddCatchAll((message, context) => { });
// Assert
act.Should().Throw<InvalidOperationException>()
.WithMessage("*already registered*");
}
[Fact]
public async Task AddCatchAll_OverwriteExistingRegistration_RoutesUpdatedHandlerAsync()
{
// Arrange
RouteBuilder routeBuilder = new(null);
routeBuilder.AddCatchAll((message, context) => context.SendMessageAsync("first"));
routeBuilder.AddCatchAll((message, context) => context.SendMessageAsync("second"), overwrite: true);
MessageRouter router = routeBuilder.Build();
TestWorkflowContext context = new("executor");
// Act
_ = await router.RouteMessageAsync(new TestPayload("hello"), context);
// Assert
context.SentMessages.Should().ContainSingle().Which.Should().Be("second");
}
[Fact]
public void AddPortHandler_WithoutExternalRequestContext_ThrowsInvalidOperationException()
{
// Arrange
RouteBuilder routeBuilder = new(null);
// Act
Action act = () => routeBuilder.AddPortHandler<string, int>("port", (response, context, cancellationToken) => default, out _);
// Assert
act.Should().Throw<InvalidOperationException>()
.WithMessage("*external request context is required*");
}
[Fact]
public async Task AddPortHandler_RoutesMatchingExternalResponseAsync()
{
// Arrange
TestExternalRequestContext externalRequestContext = new();
RouteBuilder routeBuilder = new(externalRequestContext);
HandlerInvocation invocation = new();
routeBuilder.AddPortHandler<string, int>("port", (response, context, cancellationToken) =>
{
invocation.Capture(response, context, cancellationToken);
return default;
}, out PortBinding portBinding);
await portBinding.PostRequestAsync("request", requestId: "req-1");
MessageRouter router = routeBuilder.Build();
TestWorkflowContext context = new("executor");
CancellationToken cancellationToken = new CancellationTokenSource().Token;
ExternalResponse response = externalRequestContext.PostedRequests.Single().CreateResponse(42);
// Act
CallResult? result = await router.RouteMessageAsync(response, context, cancellationToken: cancellationToken);
// Assert
externalRequestContext.RegisteredPorts.Should().ContainSingle(port => port.Id == "port");
externalRequestContext.PostedRequests.Should().ContainSingle(request => request.RequestId == "req-1");
result.Should().NotBeNull();
result!.IsSuccess.Should().BeTrue();
result.Result.Should().BeSameAs(response);
invocation.InvocationCount.Should().Be(1);
invocation.Message.Should().Be(42);
invocation.Context.Should().BeSameAs(context);
invocation.CancellationToken.Should().Be(cancellationToken);
}
[Fact]
public async Task AddPortHandler_UnknownPort_ReturnsExceptionResultAsync()
{
// Arrange
TestExternalRequestContext externalRequestContext = new();
RouteBuilder routeBuilder = new(externalRequestContext);
routeBuilder.AddPortHandler<string, int>("port", (response, context, cancellationToken) => default, out _);
MessageRouter router = routeBuilder.Build();
ExternalRequest request = ExternalRequest.Create(RequestPort.Create<string, int>("other"), "request", requestId: "req-1");
// Act
CallResult? result = await router.RouteMessageAsync(request.CreateResponse(42), new TestWorkflowContext("executor"));
// Assert
result.Should().NotBeNull();
result!.IsSuccess.Should().BeFalse();
result.Exception.Should().BeOfType<InvalidOperationException>();
result.Exception!.Message.Should().Contain("Unknown port");
}
private static void RegisterVoidHandler(RouteBuilder routeBuilder, HandlerInvocation invocation, HandlerOverload overload)
{
switch (overload)
{
case HandlerOverload.SyncWithCancellation:
routeBuilder.AddHandler<string>((message, context, cancellationToken) => invocation.Capture(message, context, cancellationToken));
break;
case HandlerOverload.SyncWithoutCancellation:
routeBuilder.AddHandler<string>((message, context) => invocation.Capture(message, context));
break;
case HandlerOverload.AsyncWithCancellation:
routeBuilder.AddHandler<string>((message, context, cancellationToken) =>
{
invocation.Capture(message, context, cancellationToken);
return default;
});
break;
case HandlerOverload.AsyncWithoutCancellation:
routeBuilder.AddHandler<string>((message, context) =>
{
invocation.Capture(message, context);
return default;
});
break;
default:
throw new ArgumentOutOfRangeException(nameof(overload));
}
}
private static void RegisterResultHandler(RouteBuilder routeBuilder, HandlerInvocation invocation, HandlerOverload overload)
{
switch (overload)
{
case HandlerOverload.SyncWithCancellation:
routeBuilder.AddHandler<string, string>((message, context, cancellationToken) =>
{
invocation.Capture(message, context, cancellationToken);
return NormalizeHandlerResult(message);
});
break;
case HandlerOverload.SyncWithoutCancellation:
routeBuilder.AddHandler<string, string>((message, context) =>
{
invocation.Capture(message, context);
return NormalizeHandlerResult(message);
});
break;
case HandlerOverload.AsyncWithCancellation:
Func<string, IWorkflowContext, CancellationToken, ValueTask<string>> asyncHandlerWithCancellation = (message, context, cancellationToken) =>
{
invocation.Capture(message, context, cancellationToken);
return new ValueTask<string>(NormalizeHandlerResult(message));
};
routeBuilder.AddHandler(asyncHandlerWithCancellation);
break;
case HandlerOverload.AsyncWithoutCancellation:
Func<string, IWorkflowContext, ValueTask<string>> asyncHandler = (message, context) =>
{
invocation.Capture(message, context);
return new ValueTask<string>(NormalizeHandlerResult(message));
};
routeBuilder.AddHandler(asyncHandler);
break;
default:
throw new ArgumentOutOfRangeException(nameof(overload));
}
}
private static void RegisterVoidCatchAll(RouteBuilder routeBuilder, HandlerInvocation invocation, HandlerOverload overload)
{
switch (overload)
{
case HandlerOverload.SyncWithCancellation:
routeBuilder.AddCatchAll((message, context, cancellationToken) => invocation.Capture(message, context, cancellationToken));
break;
case HandlerOverload.SyncWithoutCancellation:
routeBuilder.AddCatchAll((message, context) => invocation.Capture(message, context));
break;
case HandlerOverload.AsyncWithCancellation:
routeBuilder.AddCatchAll((message, context, cancellationToken) =>
{
invocation.Capture(message, context, cancellationToken);
return default;
});
break;
case HandlerOverload.AsyncWithoutCancellation:
routeBuilder.AddCatchAll((message, context) =>
{
invocation.Capture(message, context);
return default;
});
break;
default:
throw new ArgumentOutOfRangeException(nameof(overload));
}
}
private static void RegisterResultCatchAll(RouteBuilder routeBuilder, HandlerInvocation invocation, HandlerOverload overload)
{
switch (overload)
{
case HandlerOverload.SyncWithCancellation:
routeBuilder.AddCatchAll((message, context, cancellationToken) =>
{
invocation.Capture(message, context, cancellationToken);
return NormalizeCatchAllResult(message);
});
break;
case HandlerOverload.SyncWithoutCancellation:
routeBuilder.AddCatchAll((message, context) =>
{
invocation.Capture(message, context);
return NormalizeCatchAllResult(message);
});
break;
case HandlerOverload.AsyncWithCancellation:
Func<PortableValue, IWorkflowContext, CancellationToken, ValueTask<string>> asyncCatchAllWithCancellation = (message, context, cancellationToken) =>
{
invocation.Capture(message, context, cancellationToken);
return new ValueTask<string>(NormalizeCatchAllResult(message));
};
routeBuilder.AddCatchAll(asyncCatchAllWithCancellation);
break;
case HandlerOverload.AsyncWithoutCancellation:
Func<PortableValue, IWorkflowContext, ValueTask<string>> asyncCatchAll = (message, context) =>
{
invocation.Capture(message, context);
return new ValueTask<string>(NormalizeCatchAllResult(message));
};
routeBuilder.AddCatchAll(asyncCatchAll);
break;
default:
throw new ArgumentOutOfRangeException(nameof(overload));
}
}
private static bool UsesCancellationToken(HandlerOverload overload) =>
overload is HandlerOverload.SyncWithCancellation or HandlerOverload.AsyncWithCancellation;
private static string NormalizeHandlerResult(string message) => message.ToUpperInvariant();
private static string NormalizeCatchAllResult(PortableValue message) => GetPayloadValue(message).ToUpperInvariant();
private static string GetPayloadValue(PortableValue message)
{
return message.As<TestPayload>() is TestPayload payload
? payload.Value
: throw new InvalidOperationException("Expected catch-all message payload to deserialize as TestPayload.");
}
}
+1 -1
View File
@@ -16,7 +16,7 @@ Status is grouped into these buckets:
| --- | --- | --- |
| `agent-framework` | `python/` | `released` |
| `agent-framework-a2a` | `python/packages/a2a` | `beta` |
| `agent-framework-ag-ui` | `python/packages/ag-ui` | `beta` |
| `agent-framework-ag-ui` | `python/packages/ag-ui` | `rc` |
| `agent-framework-anthropic` | `python/packages/anthropic` | `beta` |
| `agent-framework-azure-contentunderstanding` | `python/packages/azure-contentunderstanding` | `alpha` |
| `agent-framework-azure-ai-search` | `python/packages/azure-ai-search` | `beta` |
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "agent-framework-ag-ui"
version = "1.0.0b260507"
version = "1.0.0rc1"
description = "AG-UI protocol integration for Agent Framework"
readme = "README.md"
license-files = ["LICENSE"]
+6
View File
@@ -34,6 +34,12 @@ devui ./agents
devui --entities my_agent.py
```
## Security Posture
DevUI is a development-only sample app, not a production hosting surface. Authentication is enabled by default.
Unauthenticated mode is allowed only on `localhost` / `127.0.0.1`; `0.0.0.0`, LAN IPs, and hostnames require
`DEVUI_AUTH_TOKEN` or `--auth-token`.
## Import Path
```python
+30 -13
View File
@@ -47,6 +47,9 @@ devui ./agents --port 8080
# → API: http://localhost:8080/v1/*
```
DevUI is auth-enabled by default. Localhost starts with a generated development token logged at startup; pass it as
`Authorization: Bearer <token>` for direct API calls.
When DevUI starts with no discovered entities, it displays a **sample entity gallery** with curated examples from the Agent Framework repository. You can download these samples, review them, and run them locally to get started quickly.
## Using MCP Tools
@@ -137,12 +140,14 @@ For convenience, DevUI provides an OpenAI Responses backend API. This means you
```bash
# Simple - use your entity name as the entity_id in metadata
curl -X POST http://localhost:8080/v1/responses \
-H "Authorization: Bearer <devui-token>" \
-H "Content-Type: application/json" \
-d @- << 'EOF'
{
"metadata": {"entity_id": "weather_agent"},
"input": "Hello world"
}
EOF
```
Or use the OpenAI Python SDK:
@@ -152,7 +157,7 @@ from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8080/v1",
api_key="not-needed" # API key not required for local DevUI
api_key="<devui-token>"
)
response = client.responses.create(
@@ -201,6 +206,7 @@ DevUI provides an **OpenAI Proxy** feature for testing OpenAI models directly th
```bash
curl -X POST http://localhost:8080/v1/responses \
-H "Authorization: Bearer <devui-token>" \
-H "X-Proxy-Backend: openai" \
-d '{"model": "gpt-4.1-mini", "input": "Hello"}'
```
@@ -214,14 +220,14 @@ devui [directory] [options]
Options:
--port, -p Port (default: 8080)
--host Host (default: 127.0.0.1)
--host Host (default: 127.0.0.1; non-loopback hosts require auth)
--headless API only, no UI
--no-open Don't automatically open browser
--instrumentation Enable OpenTelemetry instrumentation
--reload Enable auto-reload
--mode developer|user (default: developer)
--auth Enable Bearer token authentication
--auth-token Custom authentication token
--no-auth Disable auth for loopback-only local development
--auth-token Custom authentication token (required for non-loopback hosts unless DEVUI_AUTH_TOKEN is set)
```
### UI Modes
@@ -233,8 +239,8 @@ Options:
# Development
devui ./agents
# Production (user-facing)
devui ./agents --mode user --auth
# Local-only no-auth development
devui ./agents --no-auth
```
## Key Endpoints
@@ -336,28 +342,39 @@ These custom extensions are clearly namespaced and can be safely ignored by stan
## Security
DevUI is designed as a **sample application for local development** and should not be exposed to untrusted networks without proper authentication.
DevUI is designed as a **sample application for local development** and is not intended for production use. For
production, or for features beyond this sample app, build a custom interface and API server using the Agent Framework SDK.
**For production deployments:**
Auth is enabled by default. Unauthenticated mode is allowed only when DevUI is bound to `localhost` or `127.0.0.1`.
Network-reachable binds such as `0.0.0.0`, LAN IPs, and hostnames require Bearer token authentication with an explicit
token.
**For shared development hosts:**
```bash
# User mode with authentication (recommended)
devui ./agents --mode user --auth --host 0.0.0.0
# Set a token explicitly before binding beyond loopback
DEVUI_AUTH_TOKEN="<secure-dev-token>" devui ./agents --mode user --host 0.0.0.0
# Or pass the token on the command line
devui ./agents --mode user --host 0.0.0.0 --auth-token "<secure-dev-token>"
```
This restricts developer APIs (reload, deployment, entity details) and requires Bearer token authentication.
Do not use `--no-auth` with `0.0.0.0`, LAN IPs, or hostnames. That configuration fails closed before startup.
**Security features:**
- User mode restricts developer-facing APIs
- Optional Bearer token authentication via `--auth`
- Bearer token authentication is enabled by default
- Unauthenticated mode is loopback-only (`localhost` / `127.0.0.1`)
- Non-loopback binds require `DEVUI_AUTH_TOKEN` or `--auth-token`
- Only loads entities from local directories or in-memory registration
- No remote code execution capabilities
- Binds to localhost (127.0.0.1) by default
**Best practices:**
- Use `--mode user --auth` for any deployment exposed to end users
- Do not use DevUI as a production deployment surface
- Use `--mode user` plus `DEVUI_AUTH_TOKEN` or `--auth-token` for shared development hosts
- Review all agent/workflow code before running
- Only load entities from trusted sources
- Use `.env` files for sensitive credentials (never commit them)
@@ -126,29 +126,6 @@ def serve(
if not isinstance(port, int) or not (1 <= port <= 65535):
raise ValueError(f"Invalid port: {port}. Must be integer between 1 and 65535")
# Security check: warn loudly when network-exposed without authentication.
if host not in ("127.0.0.1", "localhost") and not auth_enabled:
logger.warning("WARNING: Exposing DevUI to the network with --no-auth.")
logger.warning("Anyone on your network can read agent metadata and trigger requests.")
logger.warning("Drop --no-auth and DevUI will require Bearer tokens.")
# Refuse to auto-generate a token for network-exposed binds. Auto-generated tokens
# are fine for localhost convenience; for anything else, require an explicit token.
if auth_enabled and not auth_token:
import os
env_token = os.environ.get("DEVUI_AUTH_TOKEN")
if not env_token:
is_production = (
host not in ("127.0.0.1", "localhost")
or os.environ.get("CI") == "true"
or os.environ.get("KUBERNETES_SERVICE_HOST")
)
if is_production:
logger.error("Authentication required but no token provided.")
logger.error("Set DEVUI_AUTH_TOKEN env var or pass auth_token='...' to serve().")
raise ValueError("DEVUI_AUTH_TOKEN required when host is not localhost")
# Enable instrumentation if requested
if instrumentation_enabled:
from agent_framework.observability import enable_instrumentation
@@ -81,13 +81,15 @@ Examples:
parser.add_argument(
"--no-auth",
action="store_true",
help="Disable Bearer token authentication. DevUI is auth-enabled by default; use this to opt out.",
help=(
"Disable Bearer token authentication for loopback-only local development. Non-loopback hosts require auth."
),
)
parser.add_argument(
"--auth-token",
type=str,
help="Custom Bearer token. Auto-generated and logged at startup when omitted.",
help="Custom Bearer token. Required for non-loopback hosts when DEVUI_AUTH_TOKEN is not set.",
)
parser.add_argument("--version", action="version", version=f"Agent Framework DevUI {get_version()}")
@@ -89,7 +89,7 @@ class DevServer:
mode: Server mode - 'developer' (full access, verbose errors) or 'user' (restricted APIs, generic errors)
auth_enabled: Whether to require Bearer token auth on /v1/* endpoints. Defaults to True.
auth_token: Bearer token. If None and auth_enabled, falls back to the DEVUI_AUTH_TOKEN
environment variable, then to an auto-generated token (logged at startup).
environment variable. Loopback binds may use an auto-generated token logged at startup.
"""
self.entities_dir = entities_dir
self.port = port
@@ -106,7 +106,7 @@ class DevServer:
self.ui_enabled = ui_enabled
self.mode = mode
self.auth_enabled = auth_enabled
self.auth_token = self._resolve_auth_token(auth_enabled, auth_token)
self.auth_token = self._resolve_auth_token(host, auth_enabled, auth_token)
self.executor: AgentFrameworkExecutor | None = None
self.openai_executor: OpenAIExecutor | None = None
self.deployment_manager = DeploymentManager()
@@ -118,8 +118,14 @@ class DevServer:
"""Set in-memory entities to register on startup."""
self._pending_entities = entities
_AUTH_LOOPBACK_HOSTS = frozenset({"127.0.0.1", "localhost"})
_LOOPBACK_HOSTS = frozenset({"127.0.0.1", "localhost", "[::1]", "::1"})
@classmethod
def _is_auth_loopback_host(cls, host: str) -> bool:
"""Return True when unauthenticated DevUI may be limited to local loopback."""
return host.lower() in cls._AUTH_LOOPBACK_HOSTS
def _loopback_allowed_hosts(self) -> frozenset[str] | None:
"""Return the Host-header allowlist when bound to a loopback interface, else None.
@@ -131,16 +137,25 @@ class DevServer:
return None
return self._LOOPBACK_HOSTS
@staticmethod
def _resolve_auth_token(auth_enabled: bool, auth_token: str | None) -> str | None:
@classmethod
def _resolve_auth_token(cls, host: str, auth_enabled: bool, auth_token: str | None) -> str | None:
"""Resolve the active Bearer token. Returns None when auth is disabled."""
is_loopback = cls._is_auth_loopback_host(host)
if not auth_enabled:
if not is_loopback:
raise ValueError(
"DevUI authentication cannot be disabled for non-loopback hosts. "
"Bind to 127.0.0.1/localhost for no-auth local development, or enable auth and provide "
"DEVUI_AUTH_TOKEN or auth_token for network-reachable binds."
)
return None
if auth_token:
return auth_token
env_token = os.getenv("DEVUI_AUTH_TOKEN")
if env_token:
return env_token
if not is_loopback:
raise ValueError("DEVUI_AUTH_TOKEN or auth_token is required when DevUI is bound to a non-loopback host.")
generated = secrets.token_urlsafe(32)
logger.info("=" * 70)
logger.info("DevUI authentication enabled with auto-generated token:")
+7
View File
@@ -60,6 +60,9 @@ devui
This launches the UI with all example agents/workflows at http://localhost:8080
DevUI is auth-enabled by default. Copy the generated token from startup logs and pass it as
`Authorization: Bearer <token>` for direct API calls. Use `--no-auth` only for loopback-only local testing.
## 5. What You'll See
- A web interface for testing agents interactively
@@ -74,6 +77,7 @@ You can also test via API calls:
```bash
curl -X POST http://localhost:8080/v1/responses \
-H "Authorization: Bearer <devui-token>" \
-H "Content-Type: application/json" \
-d '{
"model": "weather_agent",
@@ -86,6 +90,7 @@ curl -X POST http://localhost:8080/v1/responses \
```bash
# Create a conversation
curl -X POST http://localhost:8080/v1/conversations \
-H "Authorization: Bearer <devui-token>" \
-H "Content-Type: application/json" \
-d '{"metadata": {"agent_id": "weather_agent"}}'
@@ -93,6 +98,7 @@ curl -X POST http://localhost:8080/v1/conversations \
# Use conversation ID in requests
curl -X POST http://localhost:8080/v1/responses \
-H "Authorization: Bearer <devui-token>" \
-H "Content-Type: application/json" \
-d '{
"model": "weather_agent",
@@ -102,6 +108,7 @@ curl -X POST http://localhost:8080/v1/responses \
# Continue the conversation
curl -X POST http://localhost:8080/v1/responses \
-H "Authorization: Bearer <devui-token>" \
-H "Content-Type: application/json" \
-d '{
"model": "weather_agent",
@@ -4,8 +4,10 @@
import asyncio
import inspect
import sys
import tempfile
from pathlib import Path
from typing import Any
import pytest
from conftest import MockAgent
@@ -492,7 +494,7 @@ def test_devserver_requires_auth_by_default(monkeypatch):
def test_devserver_auth_can_be_explicitly_disabled(monkeypatch):
"""Callers can opt out of auth with auth_enabled=False (escape hatch for tests / trusted hosts)."""
"""Callers can opt out of auth on loopback (escape hatch for tests / trusted local hosts)."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
server = _server_with_mock_agent(auth_enabled=False)
@@ -504,6 +506,106 @@ def test_devserver_auth_can_be_explicitly_disabled(monkeypatch):
assert response.status_code == 200
def test_devserver_rejects_non_loopback_no_auth(monkeypatch):
"""Non-loopback binds must not be network-reachable without authentication."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
with pytest.raises(ValueError, match="authentication cannot be disabled"):
DevServer(host="0.0.0.0", auth_enabled=False)
with pytest.raises(ValueError, match="authentication cannot be disabled"):
DevServer(host="devui.example", auth_enabled=False)
def test_devserver_rejects_non_loopback_without_explicit_token(monkeypatch):
"""Network-reachable auth requires an operator-provided token, not a generated token."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
with pytest.raises(ValueError, match="DEVUI_AUTH_TOKEN or auth_token"):
DevServer(host="0.0.0.0")
def test_devserver_allows_non_loopback_with_explicit_token(monkeypatch):
"""A network-reachable bind is allowed when auth has an explicit token."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
server = DevServer(host="0.0.0.0", auth_token="s3cret")
assert server.auth_enabled is True
assert server.auth_token == "s3cret"
def test_devserver_allows_non_loopback_with_env_token(monkeypatch):
"""A network-reachable bind is allowed when auth uses DEVUI_AUTH_TOKEN."""
monkeypatch.setenv("DEVUI_AUTH_TOKEN", "env-s3cret")
server = DevServer(host="0.0.0.0")
assert server.auth_enabled is True
assert server.auth_token == "env-s3cret"
def test_devserver_allows_loopback_no_auth(monkeypatch):
"""Unauthenticated DevUI remains available for local-only development and tests."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
for host in ("127.0.0.1", "localhost"):
server = DevServer(host=host, auth_enabled=False)
assert server.auth_enabled is False
assert server.auth_token is None
def test_devserver_loopback_auth_auto_generates_token(monkeypatch):
"""Loopback auth-enabled usage may still use a generated development token."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
server = DevServer(host="127.0.0.1")
assert server.auth_enabled is True
assert server.auth_token
def test_serve_rejects_non_loopback_no_auth(monkeypatch):
"""The public serve() helper must inherit the DevServer network-auth invariant."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
with pytest.raises(ValueError, match="authentication cannot be disabled"):
agent_framework_devui.serve(entities=[], host="0.0.0.0", auth_enabled=False, ui_enabled=False)
def test_serve_rejects_non_loopback_without_explicit_token(monkeypatch):
"""serve() must not maintain a weaker generated-token path for network binds."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
with pytest.raises(ValueError, match="DEVUI_AUTH_TOKEN or auth_token"):
agent_framework_devui.serve(entities=[], host="0.0.0.0", ui_enabled=False)
def test_serve_allows_non_loopback_with_explicit_token(monkeypatch):
"""serve() accepts a network bind when an explicit token is provided."""
import uvicorn
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
run_args = {}
def fake_run(_app, *, host, port, **_kwargs):
run_args["host"] = host
run_args["port"] = port
monkeypatch.setattr(uvicorn, "run", fake_run)
agent_framework_devui.serve(
entities=[],
host="0.0.0.0",
port=9090,
auth_token="s3cret",
auto_open=False,
ui_enabled=False,
)
assert run_args == {"host": "0.0.0.0", "port": 9090}
def test_devserver_accepts_request_with_valid_bearer_token(monkeypatch):
"""When auth is on, supplying the configured Bearer token grants access."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
@@ -567,8 +669,8 @@ def test_serve_defaults_to_auth_enabled():
)
def test_cli_enables_auth_by_default_and_supports_no_auth_optout():
"""`devui ./agents` must produce auth-enabled config; `--no-auth` is the explicit escape hatch."""
def test_cli_enables_auth_by_default_and_supports_loopback_no_auth_optout():
"""`devui ./agents` must produce auth-enabled config; `--no-auth` is the loopback-only escape hatch."""
from agent_framework_devui._cli import create_cli_parser
parser = create_cli_parser()
@@ -578,3 +680,76 @@ def test_cli_enables_auth_by_default_and_supports_no_auth_optout():
optout_args = parser.parse_args(["--no-auth"])
assert optout_args.no_auth is True
help_text = parser.format_help()
assert "loopback-only" in help_text
assert "Non-loopback hosts require auth" in help_text
def _run_cli_with_fake_uvicorn(monkeypatch, tmp_path: Path, *args: str) -> dict[str, Any]:
"""Run the DevUI CLI without binding a socket."""
import uvicorn
from agent_framework_devui import _cli
run_args: dict[str, Any] = {}
def fake_run(_app, *, host, port, **_kwargs):
run_args["host"] = host
run_args["port"] = port
monkeypatch.setattr(uvicorn, "run", fake_run)
monkeypatch.setattr(sys, "argv", ["devui", str(tmp_path), "--no-open", "--headless", *args])
_cli.main()
return run_args
def test_cli_allows_loopback_no_auth_without_binding_socket(monkeypatch, tmp_path):
"""`devui --no-auth` remains valid on the default loopback host."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
run_args = _run_cli_with_fake_uvicorn(monkeypatch, tmp_path, "--no-auth")
assert run_args == {"host": "127.0.0.1", "port": 8080}
def test_cli_rejects_non_loopback_no_auth_before_binding_socket(monkeypatch, tmp_path, capsys):
"""`devui --host 0.0.0.0 --no-auth` must fail through shared server validation."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
with pytest.raises(SystemExit) as exc_info:
_run_cli_with_fake_uvicorn(monkeypatch, tmp_path, "--host", "0.0.0.0", "--no-auth")
assert exc_info.value.code == 1
assert "authentication cannot be disabled" in capsys.readouterr().err
def test_cli_rejects_non_loopback_without_explicit_token_before_binding_socket(monkeypatch, tmp_path, capsys):
"""`devui --host 0.0.0.0` must fail when neither --auth-token nor DEVUI_AUTH_TOKEN is set."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
with pytest.raises(SystemExit) as exc_info:
_run_cli_with_fake_uvicorn(monkeypatch, tmp_path, "--host", "0.0.0.0")
assert exc_info.value.code == 1
assert "DEVUI_AUTH_TOKEN or auth_token" in capsys.readouterr().err
def test_cli_allows_non_loopback_with_auth_token_without_binding_socket(monkeypatch, tmp_path):
"""`devui --host 0.0.0.0 --auth-token ...` starts with token auth enabled."""
monkeypatch.delenv("DEVUI_AUTH_TOKEN", raising=False)
run_args = _run_cli_with_fake_uvicorn(monkeypatch, tmp_path, "--host", "0.0.0.0", "--auth-token", "s3cret")
assert run_args == {"host": "0.0.0.0", "port": 8080}
def test_cli_allows_non_loopback_with_env_token_without_binding_socket(monkeypatch, tmp_path):
"""`DEVUI_AUTH_TOKEN=... devui --host 0.0.0.0` starts with token auth enabled."""
monkeypatch.setenv("DEVUI_AUTH_TOKEN", "env-s3cret")
run_args = _run_cli_with_fake_uvicorn(monkeypatch, tmp_path, "--host", "0.0.0.0")
assert run_args == {"host": "0.0.0.0", "port": 8080}
@@ -11,6 +11,7 @@ import tempfile
import threading
from collections.abc import AsyncIterable, AsyncIterator, Generator, Mapping, Sequence
from contextlib import suppress
from pathlib import Path
from typing import Protocol, cast
from agent_framework import (
@@ -205,6 +206,47 @@ class FileBasedFunctionApprovalStorage:
return await asyncio.to_thread(self._load_sync, approval_request_id)
def _checkpoint_storage_for_context(root: str, context_id: str) -> FileCheckpointStorage:
"""Build a ``FileCheckpointStorage`` for ``context_id`` rooted under ``root``.
``context_id`` originates from caller-controlled fields such as
``previous_response_id`` or from server-generated fields such as
``conversation_id`` / ``response_id``. In every case it must be treated as
an untrusted single path segment: path separators, drive letters, parent
references and similar would otherwise let the resulting directory escape
the configured checkpoint root (CWE-22). The check resolves the joined
path and verifies it stays under the resolved root before any directory is
created on disk.
"""
if not isinstance(context_id, str) or not context_id:
raise RuntimeError("Invalid checkpoint context id: must be a non-empty string.")
# Reject any segment that is not a single safe path component. This covers
# POSIX/Windows separators, NUL bytes, drive letters, and all-dot segments
# (``.``, ``..``, ``...``, ...). We deliberately do not URL-decode the id
# here: the hosting layer never decodes context ids before joining them, so
# forms such as ``%2e%2e`` are accepted as literal directory names. Do NOT
# add decoding here without re-validating after the decode -- decode-then-
# join is exactly the pattern that reintroduces traversal. We also do not
# attempt to "sanitize" by stripping characters because that can introduce
# collisions between distinct ids.
if (
"/" in context_id
or "\\" in context_id
or "\x00" in context_id
# All-dot segments (``.``, ``..``, ``...``, ...) reduce to "" after stripping dots.
or context_id.strip(".") == ""
or os.path.isabs(context_id)
or os.path.splitdrive(context_id)[0]
):
raise RuntimeError(f"Invalid checkpoint context id: {context_id!r}")
root_path = Path(root).resolve()
storage_path = (root_path / context_id).resolve()
if not storage_path.is_relative_to(root_path):
raise RuntimeError(f"Invalid checkpoint context id: {context_id!r}")
return FileCheckpointStorage(storage_path)
class ResponsesHostServer(ResponsesAgentServerHost):
"""A responses server host for an agent."""
@@ -400,7 +442,7 @@ class ResponsesHostServer(ResponsesAgentServerHost):
latest_checkpoint_id: str | None = None
restore_storage: FileCheckpointStorage | None = None
if context_id is not None:
restore_storage = FileCheckpointStorage(os.path.join(self._checkpoint_storage_path, context_id))
restore_storage = _checkpoint_storage_for_context(self._checkpoint_storage_path, context_id)
latest_checkpoint = await restore_storage.get_latest(workflow_name=self._agent.workflow.name)
if latest_checkpoint is not None:
latest_checkpoint_id = latest_checkpoint.checkpoint_id
@@ -414,7 +456,7 @@ class ResponsesHostServer(ResponsesAgentServerHost):
# supplied, restore_storage points at the *prior* response's
# directory and write_storage points at the *current* response's.
write_context_id = context.conversation_id or context.response_id
write_storage = FileCheckpointStorage(os.path.join(self._checkpoint_storage_path, write_context_id))
write_storage = _checkpoint_storage_for_context(self._checkpoint_storage_path, write_context_id)
# Multi-turn pattern: when we have a prior checkpoint, restore it
# first (drive the workflow back to idle with prior state intact),
@@ -11,7 +11,7 @@ the registered _handle_create handler.
from __future__ import annotations
import json
from collections.abc import AsyncIterator
from collections.abc import AsyncIterator, Callable
from unittest.mock import AsyncMock, MagicMock
import httpx
@@ -20,6 +20,7 @@ from agent_framework import (
AgentResponse,
AgentResponseUpdate,
Content,
FileCheckpointStorage,
HistoryProvider,
Message,
RawAgent,
@@ -2652,3 +2653,241 @@ class TestFunctionApprovalRoundTrip:
# endregion
# region Checkpoint context path validation
class TestCheckpointContextPathValidation:
"""Regression tests for the path-traversal hardening of checkpoint storage.
These tests guard against CWE-22 in the workflow hosting path. The hosting
code joins caller-supplied identifiers (``previous_response_id``) and
server-generated identifiers (``conversation_id`` / ``response_id``) under
the configured checkpoint root. Without validation, traversal segments
such as ``../../escape`` or absolute paths cause directory creation
outside the intended root.
"""
@staticmethod
def _helper() -> Callable[[str, str], FileCheckpointStorage]:
from agent_framework_foundry_hosting._responses import ( # pyright: ignore[reportPrivateUsage]
_checkpoint_storage_for_context,
)
return _checkpoint_storage_for_context
def test_valid_segment_creates_storage_under_root(self, tmp_path: Any) -> None:
helper = self._helper()
root = tmp_path / "root"
root.mkdir()
storage = helper(str(root), "resp_abc123")
assert storage.storage_path.is_dir()
assert storage.storage_path.parent == root.resolve()
@pytest.mark.parametrize(
"bad_id",
[
# Original MSRC repro: traversal embedded inside an id-shaped value.
# The 14 ``A``s pad the suffix to mimic the exact length of the
# ``api-made-dir<14-char-suffix>`` segment from the original report.
"caresp_x/../../service-data/api-made-dir" + "A" * 14,
# Variant report repros.
"../../escape",
"..",
".",
"...",
"/tmp/escape",
"/absolute/path",
"C:\\temp\\escape",
"..\\..\\escape",
"foo\\..\\bar",
"foo/bar",
"with\x00null",
"",
],
)
def test_traversal_and_separator_payloads_are_rejected(self, tmp_path: Any, bad_id: str) -> None:
helper = self._helper()
# Use a dedicated root *inside* tmp_path so we can assert that nothing
# was created anywhere under tmp_path (root, siblings, or above).
# Asserting against tmp_path.parent would be flaky under parallel test
# execution because tmp_path.parent is shared across tests.
root = tmp_path / "root"
root.mkdir()
before = sorted(p.name for p in tmp_path.iterdir())
with pytest.raises(RuntimeError):
helper(str(root), bad_id)
# No sibling/escape directory should have been created next to the root.
after = sorted(p.name for p in tmp_path.iterdir())
assert before == after, f"Unexpected filesystem artifacts created for payload {bad_id!r}"
# And nothing inside the root either.
assert list(root.iterdir()) == []
def test_non_string_context_id_is_rejected(self, tmp_path: Any) -> None:
helper = self._helper()
with pytest.raises(RuntimeError):
helper(str(tmp_path), None) # type: ignore[arg-type]
def test_url_encoded_traversal_is_treated_as_literal_segment(self, tmp_path: Any) -> None:
"""URL-encoded traversal should not decode to traversal at the filesystem layer.
The hosting layer never URL-decodes ids before using them; the helper
should accept ``%2e%2e`` as a single literal segment (no escape).
"""
helper = self._helper()
root = tmp_path / "root"
root.mkdir()
storage = helper(str(root), "%2e%2e")
assert storage.storage_path.parent == root.resolve()
assert storage.storage_path.name == "%2e%2e"
@pytest.mark.parametrize(
"context_field,bad_id",
[
# Restore sink: caller-controlled previous_response_id.
("previous_response_id", "../../escape"),
("previous_response_id", "/tmp/escape-abs"),
("previous_response_id", "caresp_x/../../service-data/api-made-dir" + "A" * 14),
# Restore sink: server-issued conversation_id (defense in depth).
("conversation_id", "../../escape"),
# Write sink: malicious response_id (defense in depth).
("response_id", "../../escape"),
],
)
async def test_handle_inner_workflow_rejects_malicious_context_id(
self, tmp_path: Any, context_field: str, bad_id: str
) -> None:
"""End-to-end: ``_handle_inner_workflow`` must reject malicious ids on
both the restore sink (``previous_response_id`` / ``conversation_id``)
and the write sink (``response_id``) without creating any directories.
"""
from unittest.mock import patch
from agent_framework import WorkflowAgent
from azure.ai.agentserver.responses import ResponseContext
from azure.ai.agentserver.responses.models import CreateResponse
# Build a mock that satisfies isinstance(agent, WorkflowAgent) and the
# constructor's "no existing checkpointing" guard.
agent = MagicMock(spec=WorkflowAgent)
agent.id = "wf-agent"
agent.name = "wf"
agent.description = ""
agent.context_providers = []
agent.workflow = MagicMock()
agent.workflow.name = "wf"
agent.workflow._runner_context.has_checkpointing = MagicMock(return_value=False)
# Constructor inspects WorkflowAgent.workflow internals; bypass setup
# by feeding a configured mock through a normal init.
server = ResponsesHostServer(agent, store=InMemoryResponseProvider())
# Re-root checkpoint storage at our isolated tmp_path so we can detect
# any escape attempt on the filesystem.
root = tmp_path / "root"
root.mkdir()
server._checkpoint_storage_path = str(root) # pyright: ignore[reportPrivateUsage]
# Build a ResponseContext with the malicious id targeting the chosen sink.
kwargs: dict[str, Any] = {
"response_id": "resp_" + "a" * 48,
"mode_flags": MagicMock(),
}
if context_field == "previous_response_id":
request = CreateResponse(model="m", input="hi", previous_response_id=bad_id)
kwargs["previous_response_id"] = bad_id
elif context_field == "conversation_id":
request = CreateResponse(model="m", input="hi")
kwargs["conversation_id"] = bad_id
else: # response_id (write sink)
request = CreateResponse(model="m", input="hi")
kwargs["response_id"] = bad_id
# Avoid invoking the real input-resolution machinery, which would need
# a configured provider; we never reach the workflow run on rejection.
with patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])):
context = ResponseContext(**kwargs)
before = sorted(p.name for p in tmp_path.iterdir())
with pytest.raises(RuntimeError, match="Invalid checkpoint context id"):
async for _ in server._handle_inner_workflow(request, context): # pyright: ignore[reportPrivateUsage]
pass
after = sorted(p.name for p in tmp_path.iterdir())
assert before == after, f"Unexpected filesystem artifacts created for {context_field}={bad_id!r}"
assert list(root.iterdir()) == [], f"Checkpoint dir created inside root for {context_field}={bad_id!r}"
@pytest.mark.parametrize(
"context_field,bad_id",
[
# Restore sink: caller-controlled previous_response_id. These are
# rejected by request validation (HTTP 400) before the checkpoint
# code is reached.
("previous_response_id", "../../escape"),
("previous_response_id", "/tmp/escape-abs"),
("previous_response_id", "caresp_x/../../service-data/api-made-dir" + "A" * 14),
# Restore sink: server-issued conversation id (defense in depth).
# Reaches the checkpoint code and is rejected there, surfacing as
# an HTTP 5xx without creating any filesystem artifacts.
("conversation", "../../escape"),
("conversation", "/tmp/escape-abs"),
],
)
async def test_malicious_context_id_rejected_e2e(self, tmp_path: Any, context_field: str, bad_id: str) -> None:
"""End-to-end (ASGI-in-process): malicious context ids must be rejected
through the full HTTP pipeline, and no checkpoint directory may be
created on disk for either the validation-layer rejection
(``previous_response_id``) or the deeper checkpoint-layer rejection
(``conversation``).
The ``response_id`` write-sink is server-generated and not reachable
via the public HTTP surface, so its defense-in-depth check is covered
by the helper-level test above.
"""
from agent_framework import WorkflowAgent
# Build a mock that satisfies isinstance(agent, WorkflowAgent) and the
# constructor's "no existing checkpointing" guard.
agent = MagicMock(spec=WorkflowAgent)
agent.id = "wf-agent"
agent.name = "wf"
agent.description = ""
agent.context_providers = []
agent.workflow = MagicMock()
agent.workflow.name = "wf"
agent.workflow._runner_context.has_checkpointing = MagicMock( # pyright: ignore[reportPrivateUsage]
return_value=False
)
server = ResponsesHostServer(agent, store=InMemoryResponseProvider())
# Re-root checkpoint storage at our isolated tmp_path so we can detect
# any escape attempt on the filesystem.
root = tmp_path / "root"
root.mkdir()
server._checkpoint_storage_path = str(root) # pyright: ignore[reportPrivateUsage]
payload: dict[str, Any] = {"model": "m", "input": "hi"}
if context_field == "previous_response_id":
payload["previous_response_id"] = bad_id
else: # conversation
payload["conversation"] = bad_id
before = sorted(p.name for p in tmp_path.iterdir())
transport = httpx.ASGITransport(app=server)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
resp = await client.post("/responses", json=payload)
after = sorted(p.name for p in tmp_path.iterdir())
# The request must not succeed; either request validation rejects it
# (4xx) or the checkpoint layer raises and the server returns 5xx.
# Either way, no successful response may be produced.
assert resp.status_code >= 400, (
f"Expected non-2xx for {context_field}={bad_id!r}, got {resp.status_code}: {resp.text[:200]}"
)
assert before == after, (
f"Unexpected filesystem artifacts under tmp_path for {context_field}={bad_id!r}: "
f"before={before} after={after}"
)
assert list(root.iterdir()) == [], f"Checkpoint directory created inside root for {context_field}={bad_id!r}"
# endregion
+2 -2
View File
@@ -178,7 +178,7 @@ requires-dist = [
[[package]]
name = "agent-framework-ag-ui"
version = "1.0.0b260507"
version = "1.0.0rc1"
source = { editable = "packages/ag-ui" }
dependencies = [
{ name = "ag-ui-protocol", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -602,7 +602,7 @@ dependencies = [
[package.metadata]
requires-dist = [
{ name = "agent-framework-core", editable = "packages/core" },
{ name = "github-copilot-sdk", marker = "python_full_version >= '3.11'", specifier = ">=1.0.0b2,<=1.0.0b2" },
{ name = "github-copilot-sdk", marker = "python_full_version >= '3.11'", specifier = "<=1.0.0b2,>=1.0.0b2" },
]
[[package]]