mirror of https://github.com/microsoft/agent-framework.git synced 2026-06-16 21:04:09 +08:00

Files

T

Eduard van Valkenburg 6acab3d1d6 Python: [BREAKING] Standardize model selection on model (#4999 )

* Refactor Anthropic model option and provider clients

Rename the Anthropic client model option from model_id to model, add provider-specific Anthropic wrappers for Foundry, Bedrock, and Vertex, and expose them through the Anthropic, Foundry, Amazon, and Google namespaces. Update core option handling, docs, samples, and tests accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix Anthropic skills sample typing

Cast the Anthropic beta client to Any in the skills sample so the pre-commit sample pyright check no longer fails on beta skills and files endpoints that are not exposed by the current SDK stubs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* undo sample mypy

* Retry CI after transient external failures

Retrigger PR validation after an unrelated Copilot review workflow SAML failure and a transient external tau2 git fetch failure in the Windows Python test setup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback on model option merging

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address Anthropic compatibility review feedback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* moved all to `model`

* fixes for azure ai search

* Python: standardize remaining sample env var names

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: fix foundry-local pyright compatibility

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updated env vars in cicd

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

6acab3d1d6 · 2026-04-01 19:00:18 +00:00

History

_tools.py

Python: Add load_dotenv() to samples for .env file support (#4043 )

2026-02-19 10:55:13 +00:00

.env.example

Python: [BREAKING] Remove deprecated Python OpenAI/Azure AI surfaces (#4990 )

2026-03-31 20:36:21 +00:00

create_workflow.py

Python: [BREAKING] Standardize model selection on model (#4999 )

2026-04-01 19:00:18 +00:00

README.md

Python: restructure: Python samples into progressive 01-05 layout (#3862 )

2026-02-12 17:36:36 +00:00

run_evaluation.py

Python: [BREAKING] Standardize model selection on model (#4999 )

2026-04-01 19:00:18 +00:00

README.md

Multi-Agent Travel Planning Workflow Evaluation

This sample demonstrates evaluating a multi-agent workflow using Azure AI's built-in evaluators. The workflow processes travel planning requests through seven specialized agents in a fan-out/fan-in pattern: travel request handler, hotel/flight/activity search agents, booking aggregator, booking confirmation, and payment processing.

Evaluation Metrics

The evaluation uses four Azure AI built-in evaluators:

Relevance - How well responses address the user query
Groundedness - Whether responses are grounded in available context
Tool Call Accuracy - Correct tool selection and parameter usage
Tool Output Utilization - Effective use of tool outputs in responses

Setup

Create a .env file with configuration as in the .env.example file in this folder.

Running the Evaluation

Execute the complete workflow and evaluation:

python run_evaluation.py

The script will:

Execute the multi-agent travel planning workflow
Display response summary for each agent
Create and run evaluation on hotel, flight, and activity search agents
Monitor progress and display the evaluation report URL