mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
Evan Mattson
0f3f4dbcaf
* Refactor events * Merge main * Fixes * Cleanup * Update samples and tests * Remove unused imports * PR feedback * Merge main. Add properties for events to help typing * Formatting * Cleanup * use builtins.type to avoid shadowing by WorkflowEvent.type attribute * Final improvements
0f3f4dbcaf
ยท
2026-02-06 07:47:20 +00:00
History
Multi-Agent Travel Planning Workflow Evaluation
This sample demonstrates evaluating a multi-agent workflow using Azure AI's built-in evaluators. The workflow processes travel planning requests through seven specialized agents in a fan-out/fan-in pattern: travel request handler, hotel/flight/activity search agents, booking aggregator, booking confirmation, and payment processing.
Evaluation Metrics
The evaluation uses four Azure AI built-in evaluators:
- Relevance - How well responses address the user query
- Groundedness - Whether responses are grounded in available context
- Tool Call Accuracy - Correct tool selection and parameter usage
- Tool Output Utilization - Effective use of tool outputs in responses
Setup
Create a .env file with configuration as in the .env.example file in this folder.
Running the Evaluation
Execute the complete workflow and evaluation:
python run_evaluation.py
The script will:
- Execute the multi-agent travel planning workflow
- Display response summary for each agent
- Create and run evaluation on hotel, flight, and activity search agents
- Monitor progress and display the evaluation report URL