Python: Add AgentLoopMiddleware for re-running agents in a loop (#6174)

* Python: Add AgentLoopMiddleware for re-running agents in a loop

Add `AgentLoopMiddleware`, an `AgentMiddleware` that re-runs the wrapped
agent in a loop. A single configurable class covers three common patterns,
each with a convenience classmethod factory:

- Ralph loop (`.ralph(...)`): no exit criteria, with feedback tracking
  (`record_feedback`/`progress`), progress injection (`inject_progress`),
  optional fresh context per iteration (`fresh_context`), and an early-stop
  completion signal (`is_complete`).
- Predicate (`.with_predicate(...)`): loop while a `should_continue` callable
  returns True (e.g. paired with `todos_remaining`/`background_tasks_running`).
- Judge (`.with_judge(...)`): a second chat client decides whether the original
  request was answered, using a `JudgeVerdict` structured-output response.

The loop also auto-resolves pending function-approval / user-input requests via
an `on_approval_request` callable (bounded by `max_approval_rounds`), and the
next iteration's input is controlled by `next_message`. Supports both streaming
and non-streaming runs.

Exports `AgentLoopMiddleware`, `JudgeVerdict`, `todos_remaining`, and
`background_tasks_running`. Adds tests, a sample, and docs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Refine AgentLoopMiddleware API and sample

- with_judge: add criteria list with {{criteria}} templating into judge
  instructions plus an agent-side instruction; add fresh_context, additional
  judge feedback relay; default judge max_iterations.
- should_continue is now required and positional; supports (bool, str|None)
  feedback tuples surfaced to next_message/record_feedback via feedback kwarg.
- Judge forwards full multi-modal request and response messages.
- Default max_iterations=10 (explicit None = unbounded); removed is_complete and
  Ralph terminology; ShouldContinueResult is a real TypeAlias.
- Sample: stream all loops, print iteration counts via injected user-block
  boundaries (robust to function calling), <role>: content formatting, per-method
  expected output, and a looping todo sample.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix CI checks for AgentLoopMiddleware

- Resolve pyright errors in _loop.py: drop the always-true final_result None
  check (the while loop always assigns it) and cast finish_reason to the
  AgentResponse constructor's expected type.
- Apply pyupgrade --py310-plus: import TypeAlias from typing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Resolve mypy/pyright disagreement on finish_reason

pyright infers AgentResponse.finish_reason as including str and rejects the
direct assignment, while mypy considers a cast redundant. Drop the cast and
suppress only pyright with a targeted reportArgumentType ignore, satisfying
both type checkers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Add todo+judge AgentLoopMiddleware sample

Add a second AgentLoopMiddleware sample that composes two criteria in one
should_continue predicate: a TodoProvider check (evaluated first) and a
report-style judge chat client (evaluated once todos are complete) that grades
the assembled report against shared requirements. Register it in the middleware
samples README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Compose todo+judge loops as two middleware

Rework the todo+judge sample to compose two AgentLoopMiddleware on the agent
itself (middleware=[judge_loop, todo_loop]) instead of a single hand-written
predicate. The inner todos_remaining loop drafts the report todo-by-todo and the
outer with_judge loop re-runs it until an editor chat client judges the report
publication-ready, reusing the built-in helpers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Reset session for fresh_context loops via snapshot/restore

AgentLoopMiddleware.fresh_context previously only reset context.messages,
so with an attached session each iteration still reloaded the local
transcript or re-threaded the service-side conversation id and the model
saw the accumulated history. Snapshot the session once before the loop
(via to_dict) and restore it (from_dict + field copy) between iterations,
so every pass starts from the pre-loop baseline. The final iteration's
pass is persisted (no restore after the terminating iteration), so a
subsequent agent.run continues from there.

Removed the obsolete warning, updated docstrings and core AGENTS.md, and
added tests: a snapshot/restore round-trip, a session-reset
streaming x fresh_context x inject_progress x store matrix across multiple
runs and loop iterations, and response_format parsing across the loop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Updated samples and docstrings

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

python/samples/02-agents/middleware/agent_loop_middleware_refinement.py

@@ -0,0 +1,121 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+import asyncio
+
+from agent_framework import Agent, AgentLoopMiddleware, AgentResponse
+from agent_framework.foundry import FoundryChatClient
+from azure.identity.aio import AzureCliCredential
+from dotenv import load_dotenv
+
+# Load environment variables from .env file
+load_dotenv()
+
+"""
+Agent Loop Middleware: refinement loop (should_continue + feedback tracking)
+
+This sample demonstrates ``AgentLoopMiddleware`` driven by a ``should_continue`` predicate. The loop
+keeps refining a candidate answer until the agent's latest response contains a completion marker. It
+also shows feedback tracking: ``record_feedback`` logs per-iteration progress that is fed into the
+next pass, ``fresh_context`` restarts each pass from the original task plus that log, and
+``max_iterations`` bounds the loop as a safety cap.
+
+``next_message`` controls the input for the next iteration (it defaults to a short "continue" nudge).
+The loop is run with streaming, so the injected messages between iterations show up as ``user``
+updates; the stream is printed as ``<role>: <content>`` lines.
+
+Environment variables:
+    FOUNDRY_PROJECT_ENDPOINT — Azure AI Foundry project endpoint URL
+    FOUNDRY_MODEL            — Model deployment name
+
+Authentication:
+    Run ``az login`` before running this sample.
+"""
+
+COMPLETE_MARKER = "<promise>COMPLETE</promise>"
+
+
+async def refinement_loop(client: FoundryChatClient) -> None:
+    """Loop while the response does not yet contain a completion marker."""
+    print("\n=== Refinement loop (should_continue marker + feedback tracking, capped at 5) ===")
+
+    # 1. ``should_continue`` keeps the loop running until the agent signals it is done by including
+    #    the completion marker in its latest response. It is called with the loop keyword args and
+    #    returns True to run the agent again.
+    def should_continue(*, last_result: AgentResponse, **kwargs: object) -> bool:
+        return COMPLETE_MARKER not in last_result.text
+
+    # 2. ``record_feedback`` captures a short progress entry each iteration. Returning a string
+    #    appends it to the log (returning None falls back to the response text). The accumulated log
+    #    is injected into the next iteration's input so the agent builds on prior work.
+    def record_feedback(*, iteration: int, last_result: AgentResponse, **kwargs: object) -> str:
+        return f"iteration {iteration}: {last_result.text.strip()[:80]}"
+
+    # 3. ``fresh_context=True`` restarts each pass from the original task plus the progress log, and
+    #    ``max_iterations`` bounds the loop as a safety cap.
+    loop = AgentLoopMiddleware(
+        should_continue,
+        max_iterations=5,
+        record_feedback=record_feedback,
+        fresh_context=True,
+    )
+
+    # 4. Attach the middleware to the agent.
+    agent = Agent(
+        client=client,
+        name="refiner",
+        instructions=(
+            "You are iteratively refining a product name for a note-taking app. Each turn, build on the "
+            "progress log: propose an improved candidate with a short reason. When you are confident the "
+            f"name is final, end your message with the exact marker {COMPLETE_MARKER}."
+        ),
+        middleware=[loop],
+    )
+
+    # 5. Run once with streaming. The middleware drives the iterations, feeding progress forward until
+    #    the agent emits the completion marker or the iteration cap is reached. Each contiguous
+    #    ``user`` block marks the boundary into the next iteration, so we count loop iterations by
+    #    those boundaries (robust to function calling, where one iteration may issue several model
+    #    calls; tool calls/results are never ``user`` updates).
+    iterations = 1
+    in_user_block = False
+    assistant_open = False
+    async for update in agent.run("Suggest a name for a note-taking app.", stream=True):
+        if update.role == "user":
+            if not in_user_block:
+                iterations += 1
+                in_user_block = True
+            assistant_open = False
+            print(f"\nuser: {update.text}", flush=True)
+            continue
+        in_user_block = False
+        if update.text:
+            if not assistant_open:
+                print("\nassistant: ", end="", flush=True)
+                assistant_open = True
+            print(update.text, end="", flush=True)
+    print(f"\n\nCompleted in {iterations} iteration(s).")
+
+
+async def main() -> None:
+    async with AzureCliCredential() as credential:
+        client = FoundryChatClient(credential=credential)
+        await refinement_loop(client)
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
+
+
+"""
+Sample output (abridged; exact text varies by model):
+
+=== Refinement loop (should_continue marker + feedback tracking, capped at 5) ===
+assistant: "QuickJot" — short and evokes fast capture.
+user: Suggest a name for a note-taking app.
+user: Progress so far:
+- iteration 1: "QuickJot" — short and evokes fast capture.
+user: Continue working on the task. If it is complete, say so.
+assistant: How about "MarginNote" — it evokes jotting ideas in the margins. <promise>COMPLETE</promise>
+
+Completed in 2 iteration(s).
+"""