mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
Eduard van Valkenburg
3139347526
* fixes Python: Add env_file_path parameter to setup_observability() similar to AzureOpenAIChatClient Fixes #2186 * WIP on updates using configure_azure_monitor * improved setup and clarity * fixed root .env.example * revert changes * updated files * updated sample * updated zero code * test fixes and fixed links * fix devui * removed planning docs * added enable method and updated readme and samples * clarified docstring * add return annotation * updated naming * update capatilized version * updated readme and some fixes * updated decorator name inline with the rest * feedback from comments addressed
3139347526
·
2025-12-16 06:56:30 +00:00
History
Self-Reflection Evaluation Sample
This sample demonstrates the self-reflection pattern using Agent Framework and Azure AI Foundry's Groundedness Evaluator. For details, see Reflexion: Language Agents with Verbal Reinforcement Learning (NeurIPS 2023).
Overview
What it demonstrates:
- Iterative self-reflection loop that automatically improves responses based on groundedness evaluation
- Batch processing of prompts from JSONL files with progress tracking
- Using
AzureOpenAIChatClientwith Azure CLI authentication - Comprehensive summary statistics and detailed result tracking
Prerequisites
Azure Resources
- Azure OpenAI: Deploy models (default: gpt-4.1 for both agent and judge)
- Azure CLI: Run
az loginto authenticate
Python Environment
pip install agent-framework-core azure-ai-projects pandas --pre
Environment Variables
# .env file
AZURE_AI_PROJECT_ENDPOINT=https://<your-ai-resource>.services.ai.azure.com/api/projects/<your-ai-project>/
Running the Sample
# Basic usage
python self_reflection.py
# With options
python self_reflection.py --input my_prompts.jsonl \
--output results.jsonl \
--max-reflections 5 \
-n 10
CLI Options:
--input,-i: Input JSONL file--output,-o: Output JSONL file--agent-model,-m: Agent model name (default: gpt-4.1)--judge-model,-e: Evaluator model name (default: gpt-4.1)--max-reflections: Max iterations (default: 3)--limit,-n: Process only first N prompts
Understanding Results
The agent iteratively improves responses:
- Generate initial response
- Evaluate groundedness (1-5 scale)
- If score < 5, provide feedback and retry
- Stop at max iterations or perfect score (5/5)
Example output:
[1/31] Processing prompt 0...
Self-reflection iteration 1/3...
Groundedness score: 3/5
Self-reflection iteration 2/3...
Groundedness score: 5/5
✓ Perfect groundedness score achieved!
✓ Completed with score: 5/5 (best at iteration 2/3)