diff --git a/docs/decisions/0001-agent-tools.md b/docs/decisions/0001-agent-tools.md
new file mode 100644
index 0000000000..e08ff21afa
--- /dev/null
+++ b/docs/decisions/0001-agent-tools.md
@@ -0,0 +1,1896 @@
+---
+# These are optional elements. Feel free to remove any of them.
+status: {proposed}
+contact: {dmytrostruk}
+date: {2025-06-23}
+deciders: {stephentoub, markwallace-microsoft, RogerBarreto, westey-m}
+consulted: {}
+informed: {}
+---
+
+# Agent Tools
+
+## Context and Problem Statement
+
+AI agents increasingly rely on diverse tools like function calling, file search, and computer use, but integrating each tool often requires custom, inconsistent implementations. A unified abstraction for tool usage is essential to simplify development, ensure consistency, and enable scalable, reliable agent performance across varied tasks.
+
+## Decision Drivers
+
+- The abstraction must provide a consistent API for all tools to reduce complexity and improve developer experience.
+- The design should allow seamless integration of new tools without significant changes to existing implementations.
+- Robust mechanisms for managing tool-specific errors and timeouts are required for reliability.
+- The abstraction should support a fallback approach to directly use unsupported or custom tools, bypassing standard abstractions when necessary.
+
+## Considered Options
+
+### Option 1: Use ChatOptions.RawRepresentationFactory for Provider-Specific Tools
+
+#### Description
+
+Utilize the existing `ChatOptions.RawRepresentationFactory` to inject provider-specific tools (e.g., for an AI provider like Foundry) without extending the `AITool` abstract class from `Microsoft.Extensions.AI`.
+
+```csharp
+ChatOptions options = new()
+{
+ RawRepresentationFactory = _ => new ResponseCreationOptions()
+ {
+ Tools = { ... }, // backend-specific tools
+ },
+};
+```
+
+#### Pros
+
+- No development work needed; leverages existing `Microsoft.Extensions.AI` functionality.
+- Flexible for integrating tools from any AI provider without modifying the `AITool`.
+- Minimal codebase changes, reducing the risk of introducing errors.
+
+#### Cons
+
+- Requires a separate mechanism to register tools, complicating the developer experience.
+- Developers must know the specific AI provider (via `IChatClient`) to configure tools, reducing abstraction.
+- Inconsistent with the `AITool` abstraction, leading to fragmented tool usage patterns.
+- Poor tool discoverability, as they are not integrated into the `AITool` ecosystem.
+
+### Option 2: Add Provider-Specific AITool-Derived Types in Provider Packages
+
+#### Description
+
+Create provider-specific tool types that inherit from the `AITool` abstract class within each AI provider’s package (e.g., a Foundry package could include Foundry-specific tools). The provider’s `IChatClient` implementation would natively recognize and process these `AITool`-derived types, eliminating the need for a separate registration mechanism.
+
+#### Pros
+
+- Integrates with the `AITool` abstract class, providing a consistent developer experience within the `Microsoft.Extensions.AI`.
+- Eliminates the need for a special registration mechanism like `RawRepresentationFactory`.
+- Enhances type safety and discoverability for provider-specific tools.
+- Aligns with the standardized interface driver by leveraging `AITool` as the base class.
+
+#### Cons
+
+- Developers must know they are targeting a specific AI provider to select the appropriate `AITool`-derived types.
+- Increases maintenance overhead for each provider’s package to support and update these tool types.
+- Leads to fragmentation, as each provider requires its own set of `AITool`-derived types.
+- Potential for duplication if multiple providers implement similar tools with different `AITool` derivatives.
+
+### Option 3: Create Generic AITool-Derived Abstractions in M.E.AI.Abstractions
+
+#### Description
+
+Develop generic tool abstractions that inherit from the `AITool` abstract class in the `M.E.AI.Abstractions` package (e.g., `HostedCodeInterpreterTool`, `HostedWebSearchTool`). These abstractions map to common tool concepts across multiple AI providers, with provider-specific implementations handled internally.
+
+#### Pros
+
+- Provides a standardized `AITool`-based interface across AI providers, improving consistency and developer experience.
+- Reduces the need for provider-specific knowledge by abstracting tool implementations.
+- Highly extensible, supporting new `AITool`-derived types for common tool concepts (e.g., server-side MCP tools).
+
+#### Cons
+
+- Complex mapping logic needed to support diverse provider implementations.
+- May not cover niche or provider-specific tools, necessitating a fallback mechanism.
+
+### Option 4: Hybrid Approach Combining Options 1, 2, and 3
+
+#### Description
+
+Implement a hybrid strategy where common tools use generic `AITool`-derived abstractions in `M.E.AI.Abstractions` (Option 3), provider-specific tools (e.g., for Foundry) are implemented as `AITool`-derived types in their respective provider packages (Option 2), and rare or unsupported tools fall back to `ChatOptions.RawRepresentationFactory` (Option 1).
+
+#### Pros
+
+- Balances developer experience and flexibility by using the best `AITool`-based approach for each tool type.
+- Supports standardized `AITool` interfaces for common tools while allowing provider-specific and breakglass mechanisms.
+- Extensible and scalable, accommodating both current and future tool requirements across AI providers.
+- Addresses ancillary and intermediate content (e.g., MCP permissions) with generic types.
+
+#### Cons
+
+- Increases complexity by managing multiple `AITool` integration approaches within the same system.
+- Requires clear documentation to guide developers on when to use each option.
+- Potential for inconsistency if boundaries between approaches are not well-defined.
+- Higher maintenance burden to support and test multiple tool integration paths.
+
+## More information
+
+### AI Agent Tool Types Availability
+
+Tool Type | Azure AI Foundry Agent Service | OpenAI Assistant API | OpenAI ChatCompletion API | OpenAI Responses API | Amazon Bedrock Agents | Google | Anthropic | Description
+-- | -- | -- | -- | -- | -- | -- | -- | --
+Function Calling | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | Enables custom, stateless functions to define specific agent behaviors.
+Code Interpreter | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | Allows agents to execute code for tasks like data analysis or problem-solving.
+Search and Retrieval | ✅ (File Search, Azure AI Search) | ✅ (File Search) | ❌ | ✅ (File Search) | ✅ (Knowledge Bases) | ✅ (Vertex AI Search) | ❌ | Enables agents to search and retrieve information from files, knowledge bases, or enterprise search systems.
+Web Search | ✅ (Bing Search) | ❌ | ✅ | ✅ | ❌ | ✅ (Google Search) | ✅ | Provides real-time access to internet-based content using search engines or web APIs for dynamic, up-to-date information.
+Remote MCP Servers | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ | Gives the model access to new capabilities via Model Context Protocol servers.
+Computer Use | ❌ | ❌ | ❌ | ✅ | ✅ (ANTHROPIC.Computer) | ❌ | ✅ | Creates agentic workflows that enable a model to control a computer interface.
+OpenAPI Spec Tool | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | Integrates existing OpenAPI specifications for service APIs.
+Stateful Functions | ✅ (Azure Functions) | ❌ | ❌ | ❌ | ✅ (AWS Lambda) | ❌ | ❌ | Supports custom, stateful functions for complex agent actions.
+Text Editor | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | Allows agents to view and modify text files for debugging or editing purposes.
+Azure Logic Apps | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | Low-code/no-code solution to add workflows to AI agents.
+Microsoft Fabric | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | Enables agents to interact with data in Microsoft Fabric for insights.
+Image Generation | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | Generates or edits images using GPT image.
+
+### API Comparison
+
+#### Function Calling
+
+ Azure AI Foundry Agent Service
+ Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/function-calling?pivots=rest
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "function",
+ "function": {
+ "description": "{string}",
+ "name": "{string}",
+ "parameters": "{JSON Schema object}"
+ }
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "tool_calls": [
+ {
+ "id": "{string}",
+ "type": "function",
+ "function": {
+ "name": "{string}",
+ "arguments": "{JSON object}",
+ }
+ }
+ ]
+ }
+ ```
+
+
+ OpenAI Assistant API
+ Source: https://platform.openai.com/docs/assistants/tools/function-calling
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "function",
+ "function": {
+ "description": "{string}",
+ "name": "{string}",
+ "parameters": "{JSON Schema object}"
+ }
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "tool_calls": [
+ {
+ "id": "{string}",
+ "type": "function",
+ "function": {
+ "name": "{string}",
+ "arguments": "{JSON object}",
+ }
+ }
+ ]
+ }
+ ```
+
+
+ OpenAI ChatCompletion API
+ Source: https://platform.openai.com/docs/guides/function-calling?api-mode=chat
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "function",
+ "function": {
+ "description": "{string}",
+ "name": "{string}",
+ "parameters": "{JSON Schema object}"
+ }
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ [
+ {
+ "id": "{string}",
+ "type": "function",
+ "function": {
+ "name": "{string}",
+ "arguments": "{JSON object}",
+ }
+ }
+ ]
+ ```
+
+
+ OpenAI Responses API
+ Source: https://platform.openai.com/docs/guides/function-calling?api-mode=responses
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "function",
+ "description": "{string}",
+ "name": "{string}",
+ "parameters": "{JSON Schema object}"
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ [
+ {
+ "id": "{string}",
+ "call_id": "{string}",
+ "type": "function_call",
+ "name": "{string}",
+ "arguments": "{JSON object}"
+ }
+ ]
+ ```
+
+
+ Amazon Bedrock Agents
+ Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax
+
+ CreateAgentActionGroup Request:
+ ```json
+ {
+ "functionSchema": {
+ "name": "{string}",
+ "description": "{string}",
+ "parameters": {
+ "type": "{string | number | integer | boolean | array}",
+ "description": "{string}",
+ "required": "{boolean}"
+ }
+ }
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "invocationInputs": [
+ {
+ "functionInvocationInput": {
+ "actionGroup": "{string}",
+ "function": "{string}",
+ "parameters": [
+ {
+ "name": "{string}",
+ "type": "{string | number | integer | boolean | array}",
+ "value": {}
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ```
+
+
+ Google
+ Source: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#rest
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "functionDeclarations": [
+ {
+ "name": "{string}",
+ "description": "{string}",
+ "parameters": "{JSON Schema object}"
+ }
+ ]
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "content": {
+ "role": "model",
+ "parts": [
+ {
+ "functionCall": {
+ "name": "{string}",
+ "args": {
+ "{argument_name}": {}
+ }
+ }
+ }
+ ]
+ }
+ }
+ ```
+
+
+ Anthropic
+ Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "name": "{string}",
+ "description": "{string}",
+ "input_schema": "{JSON Schema object}"
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "id": "{string}",
+ "model": "{string}",
+ "stop_reason": "tool_use",
+ "role": "assistant",
+ "content": [
+ {
+ "type": "text",
+ "text": "{string}"
+ },
+ {
+ "type": "tool_use",
+ "id": "{string}",
+ "name": "{string}",
+ "input": {
+ "argument_name": {}
+ }
+ }
+ ]
+ }
+ ```
+
+
+#### Commonalities
+
+- **Standardized Tool Definition**: All providers use a JSON-based structure for defining tools, including a `type` field (commonly "function") and a `function` object with `name`, `description`, and `parameters` (often following JSON Schema).
+- **Tool Call Response Structure**: Responses typically include a list of tool calls with an `id`, `type`, and details about the function called (e.g., `name` and `arguments`), enabling consistent handling of function invocations.
+- **JSON Schema for Parameters**: Parameters for functions are defined using JSON Schema objects across most providers, facilitating a unified approach to parameter validation and processing.
+- **Extensibility**: The structure allows for additional metadata or fields (e.g., `call_id`, `actionGroup`), suggesting potential for abstraction to support provider-specific extensions while maintaining core compatibility.
+
+
+
+#### Code Interpreter
+
+ Azure AI Foundry Agent Service
+ Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/code-interpreter-samples?pivots=rest-api
+
+ .NET Support: ✅
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "code_interpreter"
+ }
+ ],
+ "tool_resources": {
+ "code_interpreter": {
+ "file_ids": ["{string}"],
+ "data_sources": [
+ {
+ "type": {
+ "id_asset": "{string}",
+ "uri_asset": "{string}"
+ },
+ "uri": "{string}"
+ }
+ ]
+ }
+ }
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "tool_calls": [
+ {
+ "id": "{string}",
+ "type": "code_interpreter",
+ "code_interpreter": {
+ "input": "{string}",
+ "outputs": [
+ {
+ "type": "image",
+ "file_id": "{string}"
+ },
+ {
+ "type": "logs",
+ "logs": "{string}"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ```
+
+
+ OpenAI Assistant API
+ Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/code-interpreter-samples?pivots=rest-api
+
+ .NET Support: ✅
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "code_interpreter"
+ }
+ ],
+ "tool_resources": {
+ "code_interpreter": {
+ "file_ids": ["{string}"]
+ }
+ }
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "tool_calls": [
+ {
+ "id": "{string}",
+ "type": "code",
+ "code": {
+ "input": "{string}",
+ "outputs": [
+ {
+ "type": "logs",
+ "logs": "{string}"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ```
+
+
+ OpenAI Responses API
+ Source: https://platform.openai.com/docs/guides/tools-code-interpreter
+
+ .NET Support: ❌ (currently in development: GitHub issue)
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "code_interpreter",
+ "container": { "type": "auto" }
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ [
+ {
+ "id": "{string}",
+ "code": "{string}",
+ "type": "code_interpreter_call",
+ "status": "{string}",
+ "container_id": "{string}",
+ "results": [
+ {
+ "type": "logs",
+ "logs": "{string}"
+ },
+ {
+ "type": "files",
+ "files": [
+ {
+ "file_id": "{string}",
+ "mime_type": "{string}"
+ }
+ ]
+ }
+ ]
+ }
+ ]
+ ```
+
+
+ Amazon Bedrock Agents
+ Source: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-enable-code-interpretation.html
+
+ .NET Support: ❌ (Amazon SDK has IChatClient implementation but lacks ChatOptions.RawRepresentationFactory)
+
+ CreateAgentActionGroup Request:
+ ```json
+ {
+ "actionGroupName": "{string}",
+ "parentActionGroupSignature": "AMAZON.CodeInterpreter",
+ "actionGroupState": "ENABLED"
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "trace": {
+ "orchestrationTrace": {
+ "invocationInput": {
+ "invocationType": "ACTION_GROUP_CODE_INTERPRETER",
+ "codeInterpreterInvocationInput": {
+ "code": "{string}",
+ "files": ["{string}"]
+ }
+ },
+ "observation": {
+ "codeInterpreterInvocationOutput": {
+ "executionError": "{string}",
+ "executionOutput": "{string}",
+ "executionTimeout": "{boolean}",
+ "files": ["{string}"],
+ "metadata": {
+ "clientRequestId": "{string}",
+ "endTime": "{timestamp}",
+ "operationTotalTimeMs": "{long}",
+ "startTime": "{timestamp}",
+ "totalTimeMs": "{long}",
+ "usage": {
+ "inputTokens": "{integer}",
+ "outputTokens": "{integer}"
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ ```
+
+
+ Google
+ Source: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/code-execution#googlegenaisdk_tools_code_exec_with_txt-drest
+
+ .NET Support: ❌ (official SDK lacks IChatClient implementation.)
+
+ Message Request:
+ ```json
+ {
+ "contents": {
+ "role": "{string}",
+ "parts": {
+ "text": "{string}"
+ }
+ },
+ "tools": [
+ {
+ "codeExecution": {}
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "content": {
+ "role": "model",
+ "parts": [
+ {
+ "executableCode": {
+ "language": "{string}",
+ "code": "{string}"
+ }
+ },
+ {
+ "codeExecutionResult": {
+ "outcome": "{string}",
+ "output": "{string}"
+ }
+ }
+ ]
+ }
+ }
+ ```
+
+
+ Anthropic
+ Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/code-execution-tool
+
+
+ .NET Support: ❌
+
+ - Anthropic.SDK - uses `code_interpreter` instead of `code_execution` and lacks a possibility to specify file id.
+ - Anthropic by tryAGI - has `code_execution` implementation, but it's in beta and can't be used as a tool.
+
+
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "name": "code_execution",
+ "type": "code_execution_20250522"
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "role": "assistant",
+ "container": {
+ "id": "{string}",
+ "expires_at": "{timestamp}"
+ },
+ "content": [
+ {
+ "type": "server_tool_use",
+ "id": "{string}",
+ "name": "code_execution",
+ "input": {
+ "code": "{string}"
+ }
+ },
+ {
+ "type": "code_execution_tool_result",
+ "tool_use_id": "{string}",
+ "content": {
+ "type": "code_execution_result",
+ "stdout": "{string}",
+ "stderr": "{string}",
+ "return_code": "{integer}"
+ }
+ }
+ ]
+ }
+ ```
+
+
+#### Commonalities
+
+- **Tool Type Specification**: Providers consistently define a `code_interpreter` tool type within the `tools` array, indicating support for code execution capabilities.
+- **Input and Output Handling**: Requests include mechanisms to specify code input (e.g., `input` or `code` fields), and responses return execution outputs, such as logs or files, in a structured format.
+- **File Resource Support**: Most providers allow associating files with the code interpreter (e.g., via `file_ids` or `files`), enabling data input/output for code execution.
+- **Execution Metadata**: Responses often include metadata about the execution process (e.g., `status`, `logs`, or `executionError`), which can be abstracted for standardized error handling and result processing.
+
+
+
+#### Search and Retrieval
+
+ Azure AI Foundry Agent Service
+ Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/file-search-upload-files?pivots=rest
+
+ File Search Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "file_search"
+ }
+ ],
+ "tool_resources": {
+ "file_search": {
+ "vector_store_ids": ["{string}"],
+ "vector_stores": [
+ {
+ "name": "{string}",
+ "configuration": {
+ "data_sources": [
+ {
+ "type": {
+ "id_asset": "{string}",
+ "uri_asset": "{string}"
+ },
+ "uri": "{string}"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ }
+ }
+ ```
+
+ File Search Tool Call Response:
+ ```json
+ {
+ "tool_calls": [
+ {
+ "id": "{string}",
+ "type": "file_search",
+ "file_search": {
+ "ranking_options": {
+ "ranker": "{string}",
+ "score_threshold": "{float}"
+ },
+ "results": [
+ {
+ "file_id": "{string}",
+ "file_name": "{string}",
+ "score": "{float}",
+ "content": [
+ {
+ "text": "{string}",
+ "type": "{string}"
+ }
+ ]
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ```
+
+ Azure AI Search Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "azure_ai_search"
+ }
+ ],
+ "tool_resources": {
+ "azure_ai_search": {
+ "indexes": [
+ {
+ "index_connection_id": "{string}",
+ "index_name": "{string}",
+ "query_type": "{string}"
+ }
+ ]
+ }
+ }
+ }
+ ```
+
+ Azure AI Search Tool Call Response:
+ ```json
+ {
+ "tool_calls": [
+ {
+ "id": "{string}",
+ "type": "azure_ai_search",
+ "azure_ai_search": {} // From documentation: Reserved for future use
+ }
+ ]
+ }
+ ```
+
+
+ OpenAI Assistant API
+ Source: https://platform.openai.com/docs/assistants/tools/file-search
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "file_search"
+ }
+ ],
+ "tool_resources": {
+ "file_search": {
+ "vector_store_ids": ["string"]
+ }
+ }
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "tool_calls": [
+ {
+ "id": "{string}",
+ "type": "file_search",
+ "file_search": {
+ "ranking_options": {
+ "ranker": "{string}",
+ "score_threshold": "{float}"
+ },
+ "results": [
+ {
+ "file_id": "{string}",
+ "file_name": "{string}",
+ "score": "{float}",
+ "content": [
+ {
+ "text": "{string}",
+ "type": "{string}"
+ }
+ ]
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ```
+
+
+ OpenAI Responses API
+ Source: https://platform.openai.com/docs/api-reference/responses/create
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "file_search"
+ }
+ ],
+ "tool_resources": {
+ "file_search": {
+ "vector_store_ids": ["string"]
+ }
+ }
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "output": [
+ {
+ "id": "{string}",
+ "queries": ["{string}"],
+ "status": "{in_progress | searching | incomplete | failed | completed}",
+ "type": "file_search_call",
+ "results": [
+ {
+ "attributes": {},
+ "file_id": "{string}",
+ "filename": "{string}",
+ "score": "{float}",
+ "text": "{string}"
+ }
+ ]
+ }
+ ]
+ }
+ ```
+
+
+ Amazon Bedrock Agents
+ Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_InvokeAgent.html
+
+ Message Request:
+ ```json
+ {
+ "sessionState": {
+ "knowledgeBaseConfigurations": [
+ {
+ "knowledgeBaseId": "{string}",
+ "retrievalConfiguration": {
+ "vectorSearchConfiguration": {
+ "filter": {},
+ "implicitFilterConfiguration": {
+ "metadataAttributes": [
+ {
+ "description": "{string}",
+ "key": "{string}",
+ "type": "{string}"
+ }
+ ],
+ "modelArn": "{string}"
+ },
+ "numberOfResults": "{number}",
+ "overrideSearchType": "{string}",
+ "rerankingConfiguration": {
+ "bedrockRerankingConfiguration": {
+ "metadataConfiguration": {
+ "selectionMode": "{string}",
+ "selectiveModeConfiguration": {}
+ },
+ "modelConfiguration": {
+ "additionalModelRequestFields": {
+ "string" : "{JSON string}"
+ },
+ "modelArn": "{string}"
+ },
+ "numberOfRerankedResults": "{number}"
+ },
+ "type": "{string}"
+ }
+ }
+ }
+ }
+ ]
+ }
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "trace": {
+ "orchestrationTrace": {
+ "invocationInput": {
+ "invocationType": "KNOWLEDGE_BASE",
+ "knowledgeBaseLookupInput": {
+ "knowledgeBaseId": "{string}",
+ "text": "{string}"
+ }
+ },
+ "observation": {
+ "type": "KNOWLEDGE_BASE",
+ "knowledgeBaseLookupOutput": {
+ "retrievedReferences": [
+ {
+ "metadata": {},
+ "content": {
+ "byteContent": "{string}",
+ "row": [
+ {
+ "columnName": "{string}",
+ "columnValue": "{string}",
+ "type": "{BLOB | BOOLEAN | DOUBLE | NULL | LONG | STRING}"
+ }
+ ],
+ "text": "{string}",
+ "type": "{TEXT | IMAGE | ROW}"
+ }
+ }
+ ],
+ "metadata": {
+ "clientRequestId": "{string}",
+ "endTime": "{timestamp}",
+ "operationTotalTimeMs": "{long}",
+ "startTime": "{timestamp}",
+ "totalTimeMs": "{long}",
+ "usage": {
+ "inputTokens": "{integer}",
+ "outputTokens": "{integer}"
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ ```
+
+
+ Google
+ Source: https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/grounding-with-vertex-ai-search
+
+ Message Request:
+ ```json
+ {
+ "contents": [
+ {
+ "role": "user",
+ "parts": [
+ {
+ "text": "{string}"
+ }
+ ]
+ }
+ ],
+ "tools": [
+ {
+ "retrieval": {
+ "vertexAiSearch": {
+ "datastore": "{string}"
+ }
+ }
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "content": {
+ "role": "model",
+ "parts": [
+ {
+ "text": "{string}"
+ }
+ ]
+ },
+ "groundingMetadata": {
+ "retrievalQueries": [
+ "{string}"
+ ],
+ "groundingChunks": [
+ {
+ "retrievedContext": {
+ "uri": "{string}",
+ "title": "{string}"
+ }
+ }
+ ],
+ "groundingSupport": [
+ {
+ "segment": {
+ "startIndex": "{number}",
+ "endIndex": "{number}"
+ },
+ "segment_text": "{string}",
+ "supportChunkIndices": ["{number}"],
+ "confidenceScore": ["{number}"]
+ }
+ ]
+ }
+ }
+ ```
+
+
+#### Commonalities
+
+- **Vector Store Integration**: Providers like Azure and OpenAI use `vector_store_ids` or similar constructs to reference vector stores for file search, suggesting a common approach to retrieval-augmented generation.
+- **Search Configuration**: Requests include configurations for search (e.g., `vectorSearchConfiguration`, `ranking_options`), allowing customization of retrieval parameters like result count or ranking.
+- **Result Structure**: Responses contain a list of search results with fields like `file_id`, `score`, and `content` or `text`, enabling consistent processing of retrieved data.
+- **Metadata Inclusion**: Search responses often include metadata (e.g., `score`, `timestamp`, `usage`), which can be abstracted for unified analytics and performance tracking.
+
+
+
+#### Web Search
+
+ Azure AI Foundry Agent Service
+ Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/bing-code-samples?pivots=rest
+
+ Bing Search Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "bing_grounding",
+ "bing_grounding": {
+ "search_configurations": [
+ {
+ "connection_id": "{string}",
+ "count": "{number}",
+ "market": "{string}",
+ "set_lang": "{string}",
+ "freshness": "{string}",
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ```
+
+ Bing Search Tool Call Response:
+ ```json
+ {
+ "tool_calls": [
+ {
+ "id": "{string}",
+ "type": "function",
+ "bing_grounding": {} // From documentation: Reserved for future use
+ }
+ ]
+ }
+ ```
+
+
+ OpenAI ChatCompletion API
+ Source: https://platform.openai.com/docs/guides/tools-web-search?api-mode=chat
+
+ Message Request:
+ ```json
+ {
+ "web_search_options": {},
+ "messages": [
+ {
+ "role": "user",
+ "content": "{string}"
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ [
+ {
+ "index": 0,
+ "message": {
+ "role": "assistant",
+ "content": "{string}",
+ "annotations": [
+ {
+ "type": "url_citation",
+ "url_citation": {
+ "end_index": "{number}",
+ "start_index": "{number}",
+ "title": "{string}",
+ "url": "{string}"
+ }
+ }
+ ]
+ }
+ }
+ ]
+ ```
+
+
+ OpenAI Responses API
+ Source: https://platform.openai.com/docs/guides/tools-web-search?api-mode=responses
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "web_search_preview"
+ }
+ ],
+ "input": "{string}"
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "output": [
+ {
+ "type": "web_search_call",
+ "id": "{string}",
+ "status": "{string}"
+ },
+ {
+ "id": "{string}",
+ "type": "message",
+ "status": "{string}",
+ "role": "assistant",
+ "content": [
+ {
+ "type": "output_text",
+ "text": "{string}",
+ "annotations": [
+ {
+ "type": "url_citation",
+ "start_index": "{number}",
+ "end_index": "{string}",
+ "url": "{string}",
+ "title": "{string}"
+ }
+ ]
+ }
+ ]
+ }
+ ]
+ }
+ ```
+
+
+ Google
+ Source: https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/grounding-with-google-search
+
+ Message Request:
+ ```json
+ {
+ "contents": [
+ {
+ "role": "user",
+ "parts": [
+ {
+ "text": "{string}"
+ }
+ ]
+ }
+ ],
+ "tools": [
+ {
+ "googleSearch": {}
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "content": {
+ "role": "model",
+ "parts": [
+ {
+ "text": "{string}"
+ }
+ ]
+ },
+ "groundingMetadata": {
+ "webSearchQueries": [
+ "{string}"
+ ],
+ "searchEntryPoint": {
+ "renderedContent": "{string}"
+ },
+ "groundingChunks": [
+ {
+ "web": {
+ "uri": "{string}",
+ "title": "{string}",
+ "domain": "{string}"
+ }
+ }
+ ],
+ "groundingSupports": [
+ {
+ "segment": {
+ "startIndex": "{number}",
+ "endIndex": "{number}",
+ "text": "{string}"
+ },
+ "groundingChunkIndices": [
+ "{number}"
+ ],
+ "confidenceScores": [
+ "{number}"
+ ]
+ }
+ ],
+ "retrievalMetadata": {}
+ }
+ }
+ ```
+
+
+ Anthropic
+ Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/web-search-tool
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "name": "web_search",
+ "type": "web_search_20250305",
+ "max_uses": "{number}",
+ "allowed_domains": ["{string}"],
+ "blocked_domains": ["{string}"],
+ "user_location": {
+ "type": "approximate",
+ "city": "{string}",
+ "region": "{string}",
+ "country": "{string}",
+ "timezone": "{string}"
+ }
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "role": "assistant",
+ "content": [
+ {
+ "type": "server_tool_use",
+ "id": "{string}",
+ "name": "web_search",
+ "input": {
+ "query": "{string}"
+ }
+ },
+ {
+ "type": "web_search_tool_result",
+ "tool_use_id": "{string}",
+ "content": [
+ {
+ "type": "web_search_result",
+ "url": "{string}",
+ "title": "{string}",
+ "encrypted_content": "{string}",
+ "page_age": "{string}"
+ }
+ ]
+ },
+ {
+ "text": "{string}",
+ "type": "text",
+ "citations": [
+ {
+ "type": "web_search_result_location",
+ "url": "{string}",
+ "title": "{string}",
+ "encrypted_index": "{string}",
+ "cited_text": "{string}"
+ }
+ ]
+ }
+ ]
+ }
+ ```
+
+
+#### Commonalities
+
+- **Tool-Based Activation**: Providers define web search as a tool (e.g., `web_search`, `bing_grounding`, `googleSearch`), typically within a `tools` array, allowing standardized activation of search capabilities.
+- **Query Input**: Requests support passing a search query (e.g., via `input`, `content`, or `query`), enabling a unified interface for initiating searches.
+- **Result Annotations**: Responses include search results with metadata like `url`, `title`, and sometimes `confidenceScores` or `citations`, which can be abstracted for consistent result presentation.
+- **Grounding Metadata**: Most providers include grounding metadata (e.g., `groundingMetadata`, `annotations`), facilitating traceability and validation of search results.
+
+
+
+#### Remote MCP Servers
+
+ OpenAI Responses API
+ Source: https://platform.openai.com/docs/guides/tools-remote-mcp
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "mcp",
+ "server_label": "{string}",
+ "server_url": "{string}",
+ "require_approval": "{string}"
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "output": [
+ {
+ "id": "{string}",
+ "type": "mcp_list_tools",
+ "server_label": "{string}",
+ "tools": [
+ {
+ "name": "{string}",
+ "input_schema": "{JSON Schema object}"
+ }
+ ]
+ },
+ {
+ "id": "{string}",
+ "type": "mcp_call",
+ "approval_request_id": "{string}",
+ "arguments": "{JSON string}",
+ "error": "{string}",
+ "name": "{string}",
+ "output": "{string}",
+ "server_label": "{string}"
+ }
+ ]
+ }
+ ```
+
+
+ Google
+ Source: https://google.github.io/adk-docs/tools/mcp-tools/#using-mcp-tools-in-your-own-agent-out-of-adk-web
+
+ ```python
+ async def get_agent_async():
+ toolset = MCPToolset(
+ tool_filter=['read_file', 'list_directory'] # Optional: filter specific tools
+ connection_params=SseServerParams(url="http://remote-server:port/path", headers={...})
+ )
+
+ # Use in an agent
+ root_agent = LlmAgent(
+ model='model', # Adjust model name if needed based on availability
+ name='agent_name',
+ instruction='agent_instructions',
+ tools=[toolset], # Provide the MCP tools to the ADK agent
+ )
+ return root_agent, toolset
+ ```
+
+
+ Anthropic
+ Source: https://docs.anthropic.com/en/docs/agents-and-tools/mcp-connector
+
+ Message Request:
+ ```json
+ {
+ "messages": [
+ {
+ "role": "user",
+ "content": "{string}"
+ }
+ ],
+ "mcp_servers": [
+ {
+ "type": "url",
+ "url": "{string}",
+ "name": "{string}",
+ "tool_configuration": {
+ "enabled": true,
+ "allowed_tools": ["{string}"]
+ },
+ "authorization_token": "{string}"
+ }
+ ]
+ }
+ ```
+
+ Tool Use Response:
+ ```json
+ {
+ "type": "mcp_tool_use",
+ "id": "{string}",
+ "name": "{string}",
+ "server_name": "{string}",
+ "input": { "param1": "{object}", "param2": "{object}" }
+ }
+ ```
+
+ Tool Result Response:
+ ```json
+ {
+ "type": "mcp_tool_result",
+ "tool_use_id": "{string}",
+ "is_error": "{boolean}",
+ "content": [
+ {
+ "type": "text",
+ "text": "{string}"
+ }
+ ]
+ }
+ ```
+
+
+#### Commonalities
+
+- **Server Configuration**: Providers specify remote servers via URL and metadata (e.g., `server_url`, `url`, `name`), enabling a standardized way to connect to external MCP services.
+- **Tool Integration**: MCP tools are integrated into the `tools` or `mcp_servers` array, allowing agents to interact with remote tools in a consistent manner.
+- **Input/Output Structure**: Requests and responses include structured input (e.g., `input`, `arguments`) and output (e.g., `output`, `content`), supporting abstraction for tool execution workflows.
+- **Authorization Support**: Most providers include mechanisms for authentication (e.g., `authorization_token`, `headers`), which can be abstracted for secure communication with remote servers.
+
+
+
+#### Computer Use
+
+ OpenAI Responses API
+ Source: https://platform.openai.com/docs/guides/tools-computer-use
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "computer_use_preview",
+ "display_width": "{number}",
+ "display_height": "{number}",
+ "environment": "{browser | mac | windows | ubuntu}"
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "output": [
+ {
+ "type": "reasoning",
+ "id": "{string}",
+ "summary": [
+ {
+ "type": "summary_text",
+ "text": "{string}"
+ }
+ ]
+ },
+ {
+ "type": "computer_call",
+ "id": "{string}",
+ "call_id": "{string}",
+ "action": {
+ "type": "{click | double_click | drag | keypress | move | screenshot | scroll | type | wait}",
+ // Other properties are associated with specific action type.
+ },
+ "pending_safety_checks": [],
+ "status": "{in_progress | completed | incomplete}"
+ }
+ ]
+ }
+ ```
+
+
+ Amazon Bedrock Agents
+ Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax
+ Source: https://docs.aws.amazon.com/bedrock/latest/userguide/agent-computer-use-handle-tools.html
+
+ CreateAgentActionGroup Request:
+ ```json
+ {
+ "actionGroupName": "{string}",
+ "parentActionGroupSignature": "ANTHROPIC.Computer",
+ "actionGroupState": "ENABLED"
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "returnControl": {
+ "invocationId": "{string}",
+ "invocationInputs": [
+ {
+ "functionInvocationInput": {
+ "actionGroup": "{string}",
+ "actionInvocationType": "RESULT",
+ "agentId": "{string}",
+ "function": "{string}",
+ "parameters": [
+ {
+ "name": "{string}",
+ "type": "string",
+ "value": "{string}"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ }
+ ```
+
+
+ Anthropic
+ Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/computer-use-tool
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "computer_20250124",
+ "name": "computer",
+ "display_width_px": "{number}",
+ "display_height_px": "{number}",
+ "display_number": "{number}"
+ },
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "role": "assistant",
+ "content": [
+ {
+ "type": "tool_use",
+ "id": "{string}",
+ "name": "{string}",
+ "input": "{object}"
+ }
+ ]
+ }
+ ```
+
+
+#### Commonalities
+
+- **Tool Type Definition**: Providers define a computer use tool (e.g., `computer_use_preview`, `computer_20250124`, `ANTHROPIC.Computer`) within the `tools` array, indicating support for computer interaction capabilities.
+- **Action Specification**: Responses include actions (e.g., `click`, `keypress`, `type`) with associated parameters, enabling standardized interaction with computer environments.
+- **Environment Configuration**: Requests allow specifying the environment (e.g., `browser`, `windows`, `display_width`), which can be abstracted for cross-platform compatibility.
+- **Status Tracking**: Responses include status indicators (e.g., `status`, `pending_safety_checks`), facilitating consistent monitoring of computer use tasks.
+
+
+
+#### OpenAPI Spec Tool
+
+ Azure AI Foundry Agent Service
+ Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/openapi-spec-samples?pivots=rest-api
+ Source: https://learn.microsoft.com/en-us/rest/api/aifoundry/aiagents/run-steps/get-run-step?view=rest-aifoundry-aiagents-v1&tabs=HTTP#runstepopenapitoolcall
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "openapi",
+ "openapi": {
+ "description": "{string}",
+ "name": "{string}",
+ "auth": {
+ "type": "{string}"
+ },
+ "spec": "{OpenAPI specification object}"
+ }
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "tool_calls": [
+ {
+ "id": "{string}",
+ "type": "openapi",
+ "openapi": {} // From documentation: Reserved for future use
+ }
+ ]
+ }
+ ```
+
+
+ Amazon Bedrock Agents
+ Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax
+
+ CreateAgentActionGroup Request:
+ ```json
+ {
+ "apiSchema": {
+ "payload": "{JSON or YAML OpenAPI specification string}"
+ }
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "invocationInputs": [
+ {
+ "apiInvocationInput": {
+ "actionGroup": "{string}",
+ "apiPath": "{string}",
+ "httpMethod": "{string}",
+ "parameters": [
+ {
+ "name": "{string}",
+ "type": "{string}",
+ "value": "{string}"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ```
+
+
+#### Commonalities
+
+- **OpenAPI Specification**: Both providers support defining tools using OpenAPI specifications, either as a JSON/YAML payload or a structured `spec` object, enabling standardized API integration.
+- **Tool Type Identification**: The tool is identified as `openapi` or via an `apiSchema`, providing a clear entry point for OpenAPI-based tool usage.
+- **Parameter Handling**: Responses include parameters (e.g., `parameters`, `apiPath`, `httpMethod`) for API invocation, which can be abstracted for unified API call execution.
+
+
+
+#### Stateful Functions
+
+ Azure AI Foundry Agent Service
+ Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/azure-functions-samples?pivots=rest
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "azure_function",
+ "azure_function": {
+ "function": {
+ "name": "{string}",
+ "description": "{string}",
+ "parameters": "{JSON Schema object}"
+ },
+ "input_binding": {
+ "type": "storage_queue",
+ "storage_queue": {
+ "queue_service_endpoint": "{string}",
+ "queue_name": "{string}"
+ }
+ },
+ "output_binding": {
+ "type": "storage_queue",
+ "storage_queue": {
+ "queue_service_endpoint": "{string}",
+ "queue_name": "{string}"
+ }
+ }
+ }
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response: Not specified in the documentation.
+
+
+ Amazon Bedrock Agents
+ Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax
+
+ CreateAgentActionGroup Request:
+ ```json
+ {
+ "apiSchema": {
+ "payload": "{JSON or YAML OpenAPI specification string}"
+ }
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "invocationInputs": [
+ {
+ "apiInvocationInput": {
+ "actionGroup": "{string}",
+ "apiPath": "{string}",
+ "httpMethod": "{string}",
+ "parameters": [
+ {
+ "name": "{string}",
+ "type": "{string}",
+ "value": "{string}"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ```
+
+
+#### Commonalities
+
+- **API-Driven Interaction**: Both providers use API-based structures (e.g., `apiSchema`, `azure_function`) to define stateful functions, enabling integration with external services.
+- **Parameter Specification**: Requests include parameter definitions (e.g., `parameters`, `JSON Schema object`), supporting standardized input handling.
+
+
+
+#### Text Editor
+
+ Anthropic
+ Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/text-editor-tool
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "text_editor_20250429",
+ "name": "str_replace_based_edit_tool"
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "role": "assistant",
+ "content": [
+ {
+ "type": "tool_use",
+ "id": "{string}",
+ "name": "str_replace_based_edit_tool",
+ "input": {
+ "command": "{string}",
+ "path": "{string}"
+ }
+ }
+ ]
+ }
+ ```
+
+
+
+
+#### Microsoft Fabric
+
+ Azure AI Foundry Agent Service
+ Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/fabric?pivots=rest
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "fabric_dataagent",
+ "fabric_dataagent": {
+ "connections": [
+ {
+ "connection_id": "{string}"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response: Not specified in the documentation.
+
+
+
+
+#### Image Generation
+
+ OpenAI Responses API
+ Source: https://platform.openai.com/docs/guides/tools-image-generation
+
+ Message Request:
+ ```json
+ {
+ "tools": [
+ {
+ "type": "image_generation"
+ }
+ ]
+ }
+ ```
+
+ Tool Call Response:
+ ```json
+ {
+ "output": [
+ {
+ "type": "image_generation_call",
+ "id": "{string}",
+ "result": "{Base64 string}",
+ "status": "{string}"
+ }
+ ]
+ }
+ ```
+
+
+
+
+## Decision Outcome
+
+TBD.