mirror of https://github.com/microsoft/agent-framework.git synced 2026-06-16 21:04:09 +08:00

Files

T

Stephen Toub dc2b109b50 .NET: Upgrade to .NET 10 (#2128 )

* Upgrade to .NET 10

- Require .NET 10 SDK
- Include net10.0 assets in all assemblies
- Move net9.0-only targets to net10.0
- Update LangVersion to latest
- Remove complicated distinctions between debug target TFMs and release target TFMs
- Remove unnecessary package dependencies when built into netcoreapp
- Clean up some ifdefs
- Clean up some analyzer warnings

* Fix CI

2025-11-22 04:14:15 +00:00

2.5 KiB

Raw Permalink Blame History

Using Computer Use Tool with AI Agents

This sample demonstrates how to use the computer use tool with AI agents. The computer use tool allows agents to interact with a computer environment by viewing the screen, controlling the mouse and keyboard, and performing various actions to help complete tasks.

What this sample demonstrates

Creating agents with computer use capabilities
Using HostedComputerTool (MEAI abstraction)
Using native SDK computer use tools (ResponseTool.CreateComputerTool)
Extracting computer action information from agent responses
Handling computer tool results (text output and screenshots)
Managing agent lifecycle (creation and deletion)

Prerequisites

Before you begin, ensure you have the following prerequisites:

.NET 10 SDK or later
Azure Foundry service endpoint and deployment configured
Azure CLI installed and authenticated (for Azure credential authentication)

Note: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with az login and have access to the Azure Foundry resource. For more information, see the Azure CLI documentation.

Set the following environment variables:

$env:AZURE_FOUNDRY_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project" # Replace with your Azure Foundry resource endpoint
$env:AZURE_FOUNDRY_PROJECT_DEPLOYMENT_NAME="computer-use-preview"  # Optional, defaults to computer-use-preview

Run the sample

Navigate to the FoundryAgents sample directory and run:

cd dotnet/samples/GettingStarted/FoundryAgents
dotnet run --project .\FoundryAgents_Step15_ComputerUse

Expected behavior

The sample will:

Create two agents with computer use capabilities:
- Option 1: Using HostedComputerTool (MEAI abstraction)
- Option 2: Using native SDK computer use tools
Run the agent with a task: "I need you to help me search for 'OpenAI news'. Please type 'OpenAI news' and submit the search. Once you see search results, the task is complete."
The agent will use the computer use tool to:
- Interpret the screenshots
- Issue action requests based on the task
- Analyze the search results for "OpenAI news" from the screenshots.
Extract and display the computer actions performed
Display the results from the computer tool execution
Display the final response from the agent
Clean up resources by deleting both agents

2.5 KiB Raw Permalink Blame History