WIP: Foundry memory

2026-06-16 21:04:09 +08:00 · 2026-05-13 13:49:25 -07:00
parent 2bb7ab5c44
commit 708c0a4ab8
10 changed files with 370 additions and 1 deletions
@@ -17,7 +17,8 @@ This directory contains samples that demonstrate how to use hosted [Agent Framew
 | 7 | [Observability](responses/07_observability/) | A sample demonstrating how to enable observability for the agent deployed to Foundry. |
 | 8 | [Azure AI Search RAG](responses/08_azure_search_rag/) | An agent with Retrieval Augmented Generation (RAG) capabilities backed by Azure AI Search, grounding answers in documents indexed in a pre-provisioned search index. |
 | 9 | [Foundry Skills](responses/09_foundry_skills/) | An agent that uploads `SKILL.md` files to the Foundry Skills REST API and downloads them at startup, decoupling tone/policy guidelines from agent code. |
-| 10 | [Using deployed agent](responses/using_deployed_agent.py) | A sample demonstrating how to invoke an agent that has already been deployed to Foundry, showing how to interact with a hosted agent in code. |
+| 10 | [Foundry Memory](responses/10_foundry_memory/) | An agent with persistent semantic memory backed by an Azure AI Foundry Memory Store, using `FoundryMemoryProvider` to remember user facts across sessions. |
+| 11 | [Using deployed agent](responses/using_deployed_agent.py) | A sample demonstrating how to invoke an agent that has already been deployed to Foundry, showing how to interact with a hosted agent in code. |

 ### Invocations API

@@ -0,0 +1,8 @@
+.venv
+__pycache__
+*.pyc
+*.pyo
+*.pyd
+.Python
+.env
+provision_memory_store.py
@@ -0,0 +1,9 @@
+FOUNDRY_PROJECT_ENDPOINT="..."
+AZURE_AI_MODEL_DEPLOYMENT_NAME="..."
+# Embedding model deployment (only needed by provision_memory_store.py).
+AZURE_OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
+# Name of the Foundry Memory Store the agent should read/write to.
+FOUNDRY_MEMORY_STORE_NAME="agent_framework_memory"
+# Optional scope (e.g., user id) used to isolate memories. Falls back to the
+# per-session id when unset, which limits memories to a single session.
+FOUNDRY_MEMORY_SCOPE="user_123"
@@ -0,0 +1,16 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+COPY . user_agent/
+WORKDIR /app/user_agent
+
+RUN if [ -f requirements.txt ]; then \
+        pip install -r requirements.txt; \
+    else \
+        echo "No requirements.txt found"; \
+    fi
+
+EXPOSE 8088
+
+CMD ["python", "main.py"]
@@ -0,0 +1,126 @@
+# What this sample demonstrates
+
+An [Agent Framework](https://github.com/microsoft/agent-framework) agent with persistent semantic memory backed by an **Azure AI Foundry Memory Store**, hosted using the **Responses protocol**. The agent remembers facts the user has shared (e.g., dietary preferences, name) across sessions by retrieving and updating memories around every model invocation via `FoundryMemoryProvider`.
+
+## How It Works
+
+### Model Integration
+
+The agent uses `FoundryChatClient` from the Agent Framework to create a Responses client from the project endpoint and model deployment. `allow_preview=True` is passed so the same `AIProjectClient` can also call the preview `beta.memory_stores` API.
+
+### Memory via Foundry Memory Store
+
+`FoundryMemoryProvider` is wired into the agent as a context provider. Around each model invocation it:
+
+1. **Retrieves user-profile memories** for the configured `scope` (e.g., user id) on the first turn of a session.
+2. **Searches for contextual memories** matching the current user message and injects them into the model context.
+3. **Updates the store** with new facts inferred from the conversation.
+
+Crucially, the provider is constructed with `project_client=client.project_client` — i.e. it reuses the `AIProjectClient` that `FoundryChatClient` already created, instead of allocating a second one. This keeps a single authentication context and connection pool for both chat and memory operations.
+
+See [main.py](main.py) for the full implementation.
+
+### Agent Hosting
+
+The agent is hosted using the [Agent Framework](https://github.com/microsoft/agent-framework) with the `ResponsesHostServer`, which provisions a REST API endpoint compatible with the OpenAI Responses protocol.
+
+## Prerequisites
+
+- An Azure AI Foundry project with:
+  - A deployed chat model (e.g., `gpt-4.1-mini`)
+  - A deployed embedding model (e.g., `text-embedding-3-small`) — used by the memory store itself, not by the agent at runtime
+- Azure CLI logged in (`az login`)
+
+### Required RBAC
+
+Your identity (or the Managed Identity running the container in production) needs **Azure AI User** on the Foundry project scope. This single role covers both provisioning the memory store with `provision_memory_store.py` and reading/writing memories from `main.py`.
+
+## Provisioning the memory store (one time)
+
+[`provision_memory_store.py`](provision_memory_store.py) creates a Foundry Memory Store with the user-profile capability enabled (and chat-summary disabled) using `AIProjectClient.beta.memory_stores.create`. It is safe to re-run: if a store with the same name already exists, the script leaves it alone.
+
+From this directory, with the venv activated and `az login` done:
+
+```bash
+export FOUNDRY_PROJECT_ENDPOINT="https://<account>.services.ai.azure.com/api/projects/<project>"
+export AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4.1-mini"
+export AZURE_OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
+export FOUNDRY_MEMORY_STORE_NAME="agent_framework_memory"
+python provision_memory_store.py
+```
+
+Or in PowerShell:
+
+```powershell
+$env:FOUNDRY_PROJECT_ENDPOINT="https://<account>.services.ai.azure.com/api/projects/<project>"
+$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4.1-mini"
+$env:AZURE_OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
+$env:FOUNDRY_MEMORY_STORE_NAME="agent_framework_memory"
+python provision_memory_store.py
+```
+
+Expected output (first run):
+
+```text
+Creating memory store 'agent_framework_memory'...
+Created memory store 'agent_framework_memory' (id=memstore_...).
+```
+
+> To delete the store manually, call `project.beta.memory_stores.delete("<name>")` on an `AIProjectClient` constructed with `allow_preview=True`.
+
+## Running the Agent Host
+
+Follow the instructions in the [Running the Agent Host Locally](../../README.md#running-the-agent-host-locally) section of the README in the parent directory to run the agent host.
+
+In addition to the standard environment variables, this sample requires:
+
+```bash
+export FOUNDRY_MEMORY_STORE_NAME="agent_framework_memory"
+# Optional — defaults to "user_123" in main.py if unset.
+export FOUNDRY_MEMORY_SCOPE="user_123"
+```
+
+Or in PowerShell:
+
+```powershell
+$env:FOUNDRY_MEMORY_STORE_NAME="agent_framework_memory"
+$env:FOUNDRY_MEMORY_SCOPE="user_123"
+```
+
+You can also place these in a `.env` file next to `main.py` — see [`.env.example`](.env.example).
+
+## Interacting with the agent
+
+> Depending on how you run the agent host, you can invoke the agent using `curl` (`Invoke-WebRequest` in PowerShell) or `azd`. Please refer to the [parent README](../../README.md) for more details.
+
+Send a POST request to the server with a JSON body containing an `"input"` field to interact with the agent. The first request seeds a memory; subsequent requests (especially in new sessions) should be able to recall it because memories are scoped to `FOUNDRY_MEMORY_SCOPE`, not to a particular session.
+
+```bash
+# 1. Tell the agent something to remember.
+curl -X POST http://localhost:8088/responses -H "Content-Type: application/json" \
+  -d '{"input": "I prefer dark roast coffee and I am allergic to nuts."}'
+
+# Wait a few seconds for the memory to be stored, then start a fresh conversation:
+curl -X POST http://localhost:8088/responses -H "Content-Type: application/json" \
+  -d '{"input": "Can you recommend a coffee and a snack for me?"}'
+
+curl -X POST http://localhost:8088/responses -H "Content-Type: application/json" \
+  -d '{"input": "What do you remember about my preferences?"}'
+```
+
+The agent's answers to the follow-up turns should reflect the preferences shared in the first turn, even though `default_options={"store": False}` is set (so the Responses service is not persisting conversation history) — the recall is coming exclusively from the Foundry Memory Store.
+
+## Deploying the Agent to Foundry
+
+To host the agent on Foundry, follow the instructions in the [Deploying the Agent to Foundry](../../README.md#deploying-the-agent-to-foundry) section of the README in the parent directory.
+
+When deploying, make sure `FOUNDRY_MEMORY_STORE_NAME` and `FOUNDRY_MEMORY_SCOPE` are set in your `azd` environment so they get injected into the hosted container per [`agent.manifest.yaml`](agent.manifest.yaml):
+
+```bash
+azd env set FOUNDRY_MEMORY_STORE_NAME "agent_framework_memory"
+azd env set FOUNDRY_MEMORY_SCOPE "user_123"
+```
+
+If these are not set, running `azd ai agent init -m <agent-manifest.yaml>` will prompt you to enter them interactively.
+
+The deployed agent's Managed Identity needs **Azure AI User** on the Foundry project to read and write memories at runtime. Make sure you have run `provision_memory_store.py` against the same Foundry project before deploying — otherwise the agent will fail on the first turn when it tries to read from a non-existent store.
@@ -0,0 +1,38 @@
+name: agent-framework-agent-foundry-memory-responses
+description: >
+  An Agent Framework agent with persistent semantic memory backed by an
+  Azure AI Foundry Memory Store. Uses FoundryMemoryProvider to retrieve and
+  store memories around each model invocation, allowing the agent to remember
+  facts about a user across sessions.
+metadata:
+  tags:
+    - Agent Framework
+    - AI Agent Hosting
+    - Azure AI AgentServer
+    - Responses Protocol
+    - Foundry Memory
+template:
+  name: agent-framework-agent-foundry-memory-responses
+  kind: hosted
+  protocols:
+    - protocol: responses
+      version: 1.0.0
+  environment_variables:
+    - name: AZURE_AI_MODEL_DEPLOYMENT_NAME
+      value: "{{AZURE_AI_MODEL_DEPLOYMENT_NAME}}"
+    - name: FOUNDRY_MEMORY_STORE_NAME
+      value: "{{FOUNDRY_MEMORY_STORE_NAME}}"
+    - name: FOUNDRY_MEMORY_SCOPE
+      value: "{{FOUNDRY_MEMORY_SCOPE}}"
+parameters:
+  properties:
+    - name: FOUNDRY_MEMORY_STORE_NAME
+      secret: false
+      description: The name of the pre-provisioned Foundry Memory Store the agent will use (e.g., agent_framework_memory)
+    - name: FOUNDRY_MEMORY_SCOPE
+      secret: false
+      description: Scope (e.g., user id) used to isolate memories within the store (e.g., user_123)
+resources:
+  - kind: model
+    id: gpt-4.1-mini
+    name: AZURE_AI_MODEL_DEPLOYMENT_NAME
@@ -0,0 +1,16 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/ContainerAgent.yaml
+kind: hosted
+name: agent-framework-agent-foundry-memory-responses
+protocols:
+  - protocol: responses
+    version: 1.0.0
+resources:
+  cpu: "0.25"
+  memory: "0.5Gi"
+environment_variables:
+  - name: AZURE_AI_MODEL_DEPLOYMENT_NAME
+    value: ${AZURE_AI_MODEL_DEPLOYMENT_NAME}
+  - name: FOUNDRY_MEMORY_STORE_NAME
+    value: ${FOUNDRY_MEMORY_STORE_NAME}
+  - name: FOUNDRY_MEMORY_SCOPE
+    value: ${FOUNDRY_MEMORY_SCOPE}
@@ -0,0 +1,74 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+"""Foundry Memory hosted agent sample.
+
+This agent uses :class:`FoundryMemoryProvider` to give an otherwise stateless
+hosted agent persistent, semantic memory backed by an Azure AI Foundry
+Memory Store. The store itself is provisioned once via
+``provision_memory_store.py`` and its name is passed in through the
+``FOUNDRY_MEMORY_STORE_NAME`` environment variable.
+
+Unlike the standalone ``azure_ai_foundry_memory.py`` sample, here we construct
+the :class:`FoundryChatClient` first and then reuse its underlying
+``AIProjectClient`` for the memory provider, so both share a single client
+instance and authentication context.
+"""
+
+import asyncio
+import os
+
+from agent_framework import Agent
+from agent_framework.foundry import FoundryChatClient, FoundryMemoryProvider
+from agent_framework_foundry_hosting import ResponsesHostServer
+from azure.identity.aio import DefaultAzureCredential
+from dotenv import load_dotenv
+
+load_dotenv()
+
+
+async def main() -> None:
+    # The chat client owns the AIProjectClient. ``allow_preview=True`` is required
+    # so the same client can call the preview ``beta.memory_stores`` API used by
+    # FoundryMemoryProvider.
+    client = FoundryChatClient(
+        project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
+        model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
+        credential=DefaultAzureCredential(),
+        allow_preview=True,
+    )
+
+    # Reuse the project_client that FoundryChatClient just created, instead of
+    # constructing a second one for the memory provider.
+    memory_provider = FoundryMemoryProvider(
+        project_client=client.project_client,
+        memory_store_name=os.environ["FOUNDRY_MEMORY_STORE_NAME"],
+        # Scope namespaces memories (e.g., per end-user). When unset, the
+        # provider falls back to the session id, which limits memories to a
+        # single session.
+        scope=os.environ.get("FOUNDRY_MEMORY_SCOPE", "user_123"),
+        # In production, leave update_delay at its default to batch updates and
+        # reduce cost. We use 0 here so memories are written immediately, which
+        # makes the sample easier to demo.
+        update_delay=0,
+    )
+
+    agent = Agent(
+        client=client,
+        instructions=(
+            "You are a helpful assistant that remembers facts the user has shared "
+            "across conversations. Relevant memories from previous interactions are "
+            "automatically provided to you in the system context. Use them when "
+            "answering, and acknowledge when you are relying on remembered facts."
+        ),
+        context_providers=[memory_provider],
+        # History will be managed by the hosting infrastructure, thus there
+        # is no need to store history by the service. Learn more at:
+        # https://developers.openai.com/api/reference/resources/responses/methods/create
+        default_options={"store": False},
+    )
+    server = ResponsesHostServer(agent)
+    await server.run_async()
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
@@ -0,0 +1,78 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+"""Provision the Azure AI Foundry Memory Store used by this sample.
+
+Creates the memory store named by ``FOUNDRY_MEMORY_STORE_NAME`` if it does not
+already exist. The store is configured with the user-profile capability so the
+agent can remember stable facts about a user across sessions; chat-summary is
+disabled to keep the demo focused on durable preferences. Safe to re-run: if a
+store with the same name already exists, the script leaves it alone.
+
+Usage (from this directory, with the venv activated and ``az login`` done):
+
+    python provision_memory_store.py
+
+Required env vars (also read from a local ``.env`` file if present):
+
+    FOUNDRY_PROJECT_ENDPOINT          e.g. https://<account>.services.ai.azure.com/api/projects/<project>
+    AZURE_AI_MODEL_DEPLOYMENT_NAME    Chat model deployment used by the memory store
+    AZURE_OPENAI_EMBEDDING_MODEL      Embedding model deployment used by the memory store
+    FOUNDRY_MEMORY_STORE_NAME         Name of the memory store to create
+
+Your identity needs ``Azure AI User`` on the Foundry project scope.
+"""
+
+import asyncio
+import os
+
+from azure.ai.projects.aio import AIProjectClient
+from azure.ai.projects.models import (
+    MemoryStoreDefaultDefinition,
+    MemoryStoreDefaultOptions,
+)
+from azure.core.exceptions import ResourceNotFoundError
+from azure.identity.aio import DefaultAzureCredential
+from dotenv import load_dotenv
+
+
+async def main() -> None:
+    load_dotenv()
+
+    endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"]
+    memory_store_name = os.environ["FOUNDRY_MEMORY_STORE_NAME"]
+    chat_model = os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"]
+    embedding_model = os.environ["AZURE_OPENAI_EMBEDDING_MODEL"]
+
+    async with (
+        DefaultAzureCredential() as credential,
+        AIProjectClient(endpoint=endpoint, credential=credential, allow_preview=True) as project,
+    ):
+        try:
+            existing = await project.beta.memory_stores.get(name=memory_store_name)
+            print(f"Memory store '{existing.name}' already exists (id={existing.id}); leaving as-is.")
+            return
+        except ResourceNotFoundError:
+            pass
+
+        print(f"Creating memory store '{memory_store_name}'...")
+        definition = MemoryStoreDefaultDefinition(
+            chat_model=chat_model,
+            embedding_model=embedding_model,
+            options=MemoryStoreDefaultOptions(
+                chat_summary_enabled=False,
+                user_profile_enabled=True,
+                user_profile_details=(
+                    "Avoid irrelevant or sensitive data, such as age, financials, precise location, and credentials"
+                ),
+            ),
+        )
+        created = await project.beta.memory_stores.create(
+            name=memory_store_name,
+            description="Memory store for the Agent Framework foundry-hosted memory sample",
+            definition=definition,
+        )
+        print(f"Created memory store '{created.name}' (id={created.id}).")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
@@ -0,0 +1,3 @@
+agent-framework
+agent-framework-foundry-hosting
+azure-ai-projects