agent/agent-framework
2
0
Fork 0
mirror of https://github.com/microsoft/agent-framework.git synced 2026-06-16 21:04:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

update link (#1051)

Browse Source
This commit is contained in:
Eric Zhu
2025-10-01 11:18:11 +00:00
committed by GitHub
parent 5f1417ab94
commit 08b602c290
4 changed files with 33 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/packages/lab/tau2/README.md
+19 -17
View File
@@ -13,20 +13,22 @@ Each evaluation runs a multi-turn conversation where the user simulator presents
## Supported Domains
| Domain | Status | Description |
|--------|--------|-------------|
| **airline** | ✅ Supported | Customer service for airline booking, changes, and support |
| **retail** | 🚧 In Development | E-commerce customer support scenarios |
| **telecom** | 🚧 In Development | Telecommunications service support |
| Domain | Status | Description |
| ----------- | ----------------- | ---------------------------------------------------------- |
| **airline** | ✅ Supported | Customer service for airline booking, changes, and support |
| **retail** | 🚧 In Development | E-commerce customer support scenarios |
| **telecom** | 🚧 In Development | Telecommunications service support |
*Note: Currently only the airline domain is fully supported.*
_Note: Currently only the airline domain is fully supported._
## Installation
Install the agent-framework-lab package with TAU2 dependencies:
Install from source with TAU2 dependencies:
```bash
pip install "agent-framework-lab[tau2]"
git clone https://github.com/microsoft/agent-framework.git
cd agent-framework/python/packages/lab
pip install -e ".[tau2]"
```
Download data from [Tau2-Bench](https://github.com/sierra-research/tau2-bench):
@@ -104,15 +106,15 @@ python samples/run_benchmark.py --max-steps 20
The following results are reproduced from our implementation of τ²-bench with `samples/run_benchmark.py`. It shows the average success rate over the dataset of 50 tasks.
| Agent Model | User Model | Success Rate |
|-------------|------------|----------|
| gpt-5 | gpt-4.1 | 62.0% |
| gpt-5-mini | gpt-4.1 | 52.0% |
| gpt-4.1 | gpt-4.1 | 60.0% |
| gpt-4.1-mini | gpt-4.1 | 50.0% |
| gpt-4.1 | gpt-4o-mini | 42.0% |
| gpt-4o | gpt-4.1 | 42.0% |
| gpt-4o-mini | gpt-4.1 | 26.0% |
| Agent Model | User Model | Success Rate |
| ------------ | ----------- | ------------ |
| gpt-5 | gpt-4.1 | 62.0% |
| gpt-5-mini | gpt-4.1 | 52.0% |
| gpt-4.1 | gpt-4.1 | 60.0% |
| gpt-4.1-mini | gpt-4.1 | 50.0% |
| gpt-4.1 | gpt-4o-mini | 42.0% |
| gpt-4o | gpt-4.1 | 42.0% |
| gpt-4o-mini | gpt-4.1 | 26.0% |
## Advanced Usage