Python: Added Ollama example using OpenAIChatClient (#1100)

* Added Ollama example using OpenAIChatClient

* Small improvement
This commit is contained in:
Dmytro Struk
2025-10-02 08:14:29 -07:00
committed by GitHub
Unverified
parent 79def868b6
commit c7f20c8ec5
4 changed files with 114 additions and 1 deletions
+3
View File
@@ -21,6 +21,9 @@ COPILOTSTUDIOAGENT__AGENTAPPID=""
# Anthropic
ANTHROPIC_API_KEY=""
ANTHROPIC_MODEL=""
# Ollama
OLLAMA_ENDPOINT=""
OLLAMA_MODEL=""
# Observability
ENABLE_OTEL=true
ENABLE_SENSITIVE_DATA=true
@@ -1,6 +1,6 @@
# Anthropic Examples
This folder contains examples demonstrating how to use Anthropic's Claude models with the Agent Framework through the OpenAI Chat Client interface.
This folder contains examples demonstrating how to use Anthropic's Claude models with the Agent Framework.
## Examples
@@ -0,0 +1,30 @@
# Ollama Examples
This folder contains examples demonstrating how to use Ollama models with the Agent Framework.
## Prerequisites
1. **Install Ollama**: Download and install Ollama from [ollama.com](https://ollama.com/)
2. **Start Ollama**: Ensure Ollama is running on your local machine
3. **Pull a model**: Run `ollama pull mistral` (or any other model you prefer that supports function calling)
## Examples
| File | Description |
|------|-------------|
| [`ollama_with_openai_chat_client.py`](ollama_with_openai_chat_client.py) | Demonstrates how to configure OpenAI Chat Client to use local Ollama models. Shows both streaming and non-streaming responses with tool calling capabilities. |
## Configuration
The examples use environment variables for configuration:
### Environment Variables
Set the following environment variables before running the examples:
- `OLLAMA_ENDPOINT`: The base URL for your Ollama server
- Example: `export OLLAMA_ENDPOINT="http://localhost:11434/v1/"`
- `OLLAMA_MODEL`: The model name to use
- Example: `export OLLAMA_MODEL="mistral"`
- Must be a model you have pulled with Ollama
@@ -0,0 +1,80 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from random import randint
from typing import Annotated
from agent_framework.openai import OpenAIChatClient
"""
Ollama with OpenAI Chat Client Example
This sample demonstrates using Ollama models through OpenAI Chat Client by
configuring the base URL to point to your local Ollama server for local AI inference.
Ollama allows you to run large language models locally on your machine.
Environment Variables:
- OLLAMA_ENDPOINT: The base URL for your Ollama server (e.g., "http://localhost:11434/v1/")
- OLLAMA_MODEL: The model name to use (e.g., "mistral", "llama3.2", "phi3")
"""
def get_weather(
location: Annotated[str, "The location to get the weather for."],
) -> str:
"""Get the weather for a given location."""
conditions = ["sunny", "cloudy", "rainy", "stormy"]
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
async def non_streaming_example() -> None:
"""Example of non-streaming response (get the complete result at once)."""
print("=== Non-streaming Response Example ===")
agent = OpenAIChatClient(
base_url=os.getenv("OLLAMA_ENDPOINT"),
model_id=os.getenv("OLLAMA_MODEL"),
).create_agent(
name="WeatherAgent",
instructions="You are a helpful weather agent.",
tools=get_weather,
)
query = "What's the weather like in Seattle?"
print(f"User: {query}")
result = await agent.run(query)
print(f"Result: {result}\n")
async def streaming_example() -> None:
"""Example of streaming response (get results as they are generated)."""
print("=== Streaming Response Example ===")
agent = OpenAIChatClient(
base_url=os.getenv("OLLAMA_ENDPOINT"),
model_id=os.getenv("OLLAMA_MODEL"),
).create_agent(
name="WeatherAgent",
instructions="You are a helpful weather agent.",
tools=get_weather,
)
query = "What's the weather like in Portland?"
print(f"User: {query}")
print("Agent: ", end="", flush=True)
async for chunk in agent.run_stream(query):
if chunk.text:
print(chunk.text, end="", flush=True)
print("\n")
async def main() -> None:
print("=== Ollama with OpenAI Chat Client Agent Example ===")
await non_streaming_example()
await streaming_example()
if __name__ == "__main__":
asyncio.run(main())