mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
Python: Adding AgentFileStore and FileAccessProvider to support file access operations. (#6099)
* Adding AgentFileStore and FileAccessProvider to support file ased operations for agents. * Address PR review feedback on FileAccessProvider - Probe symlinks on the unresolved candidate path so in-root symlinks cannot silently pass and out-of-root symlinks surface the correct error message. - Validate matching_lines elements in FileSearchResult.from_dict and raise a clean ValueError for non-mapping entries. - Cap search regex pattern length (256 chars) via a new _compile_search_regex helper to mitigate ReDoS, and surface the cap in the file_access_search_files tool description. - Skip non-UTF-8 files during filesystem search instead of aborting the entire directory walk. - Replace the module-scope trailing string in the data-processing sample with comments to avoid Ruff B018. - Remove the checked-in working/region_totals.md sample artifact so the save flow works from a clean checkout. - Expand the Windows stdout reconfiguration comment in task_runner.py for clarity. - Add tests for invalid/oversize regex, non-UTF-8 file search, and in-root symlink rejection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix mypy redundant-cast in FileSearchResult.from_dict Use cast(list[object], ...) instead of cast(list[Any], ...) so the cast represents a real type change (lists are invariant) and is no longer flagged by mypy as redundant, while still satisfying pyright's reportUnknownVariableType. Matches the existing pattern in _memory.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Tighten path normalization and directory resolution in FileAccess - _normalize_relative_path now strips surrounding whitespace up front so leading/trailing spaces never leak into file segments, and rejects trailing path separators for file paths so 'foo/' is no longer silently coerced to 'foo'. - FileSystemAgentFileStore._resolve_safe_directory_path normalizes with is_directory=True and maps an empty normalized result to the root. This matches InMemoryAgentFileStore so whitespace-only directory inputs resolve to the root instead of raising. - Added tests for whitespace stripping, trailing-separator rejection, and whitespace-only directory listing on the filesystem store. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Harden FileAccess search and atomic save in store API - Add wall-clock timeout (10s) around regex scans so a pathological pattern (e.g. `(a+)+`) below the length cap cannot stall the event loop. - Offload the InMemoryAgentFileStore regex scan to a worker thread, matching the filesystem store. - Fail closed when `Path.is_symlink` raises during the safe-path probe so a permission error cannot silently bypass the symlink/reparse-point rejection. - Add `overwrite: bool = True` to `AgentFileStore.write_file`; the in-memory store performs the check under the existing lock and the filesystem store uses `open(mode='x')` so concurrent callers cannot race past `overwrite=False`. - `file_access_save_file` now relies on the atomic store call instead of a separate `file_exists` round-trip. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix Python 3.10 timeout handling and add directory arg to list/search tools - Catch asyncio.TimeoutError in _run_search_with_timeout. In Python 3.10 asyncio.wait_for raises asyncio.exceptions.TimeoutError, which is distinct from the builtin TimeoutError (the two were unified in 3.11). Catching the asyncio alias works on every supported version. - Add an optional directory parameter to file_access_list_files and file_access_search_files so agents can enumerate / scope searches to nested folders, not just the store root. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address FileAccess review feedback: case, errors, signal, TOCTOU - InMemoryAgentFileStore now stores (display_name, content) so list_files and search_files return the original-case names callers wrote, matching the behaviour of FileSystemAgentFileStore on case-preserving filesystems and removing the silent in-memory vs. on-disk contract divergence. - FileSystemAgentFileStore.read_file raises ValueError instead of letting UnicodeDecodeError bubble for binary / non-UTF-8 input, restoring symmetry with search_files (which still skips) and giving the tool layer a recoverable type to translate. - Tool wrappers now catch ValueError and OSError around every operation and surface them as readable strings, so 'you used ..' and 'the file already exists' are both reported to the model the same way instead of the former crashing out as an unhandled exception. - _search_files_sync logs per skipped non-UTF-8 file at WARNING and an aggregate INFO summary so operators can distinguish 'no matches' from 'half the corpus was unreadable'. - FileSystemAgentFileStore softens its docstrings to acknowledge the inherent probe-then-open TOCTOU window. On POSIX both read and write now pass O_NOFOLLOW so the kernel refuses if the leaf segment becomes a symlink between the probe and the open. Windows has no equivalent flag; the limitation is documented. - Tests cover: case preservation on list/search, ValueError on non-UTF-8 read at the store and tool layer, tool-layer string responses for path-traversal and oversized-regex inputs, search-skip log output, symlink rejection on delete/search/list, and symlinked intermediate directory rejection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address FileAccess nit comments: docstrings, enumerate, opt-in delete approval - Expand FileSearchMatch/FileSearchResult.to_dict docstrings to explain why the override is needed (__slots__ defeats the mixin's __dict__ iteration) and why exclude/exclude_none are accepted-but-ignored (mixin signature compatibility for callers like to_json). - Use enumerate(lines, start=1) in _search_file_content so the +1 below is no longer needed; rename loop variable to line_number for clarity. - Add opt-in require_delete_approval: bool = False on FileAccessProvider. When True, file_access_delete_file is registered with approval_mode 'always_require' so the host must approve every delete. Default False preserves current behaviour and matches the .NET reference, but deployments that want a safer-by-default posture can enable it. - Add tests covering both delete approval modes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * FileAccess: require delete approval by default Flip the default for FileAccessProvider(require_delete_approval=...) from False to True so destructive deletes are gated by host approval out of the box. Callers that want the previous autonomous behaviour (which matches the .NET reference) can pass require_delete_approval=False. Tests updated accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fixing linkinspector by installing Chrome for puppeteer first. --------- Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
committed by
GitHub
Unverified
parent
0578f4c910
commit
b000a2cf51
@@ -76,6 +76,14 @@ agent_framework/
|
||||
- **`SkillScriptRunner`** - Protocol for file-based script execution. Any callable matching `(skill, script, args) -> Any` satisfies it. Code-defined scripts do not use a runner.
|
||||
- **`SkillsProvider`** - Context provider (extends `ContextProvider`) that discovers file-based skills from `SKILL.md` files and/or accepts code-defined `Skill` instances. Follows progressive disclosure: advertise → load → read resources / run scripts.
|
||||
|
||||
### File Access Harness (`_harness/_file_access.py`)
|
||||
|
||||
- **`AgentFileStore`** - Abstract async store backing the file-access harness. Implementations expose `write_file`, `read_file`, `delete_file`, `list_files`, `file_exists`, `search_files`, and `create_directory` over forward-slash relative paths.
|
||||
- **`InMemoryAgentFileStore`** - Dict-backed store suitable for tests and lightweight scenarios.
|
||||
- **`FileSystemAgentFileStore`** - Disk-backed store rooted under a configurable directory. Enforces relative-path normalization, root containment, and rejects symlink/reparse-point segments to prevent escape.
|
||||
- **`FileSearchResult`** / **`FileSearchMatch`** - `SerializationMixin` DTOs returned by `search_files`, carrying the matching file name, a context snippet, and the matching lines with 1-based line numbers.
|
||||
- **`FileAccessProvider`** - `ContextProvider` that adds shared file-access tools (`file_access_save_file`, `file_access_read_file`, `file_access_delete_file`, `file_access_list_files`, `file_access_search_files`) plus default usage instructions to each invocation. Unlike `MemoryContextProvider`, the store is intentionally shared across sessions and agents.
|
||||
|
||||
### Workflows (`_workflows/`)
|
||||
|
||||
- **`Workflow`** - Graph-based workflow definition
|
||||
|
||||
@@ -90,6 +90,16 @@ from ._harness._background_agents import (
|
||||
BackgroundTaskInfo,
|
||||
BackgroundTaskStatus,
|
||||
)
|
||||
from ._harness._file_access import (
|
||||
DEFAULT_FILE_ACCESS_INSTRUCTIONS,
|
||||
DEFAULT_FILE_ACCESS_SOURCE_ID,
|
||||
AgentFileStore,
|
||||
FileAccessProvider,
|
||||
FileSearchMatch,
|
||||
FileSearchResult,
|
||||
FileSystemAgentFileStore,
|
||||
InMemoryAgentFileStore,
|
||||
)
|
||||
from ._harness._memory import (
|
||||
DEFAULT_MEMORY_SOURCE_ID,
|
||||
MemoryContextProvider,
|
||||
@@ -309,6 +319,8 @@ __all__ = [
|
||||
"APP_INFO",
|
||||
"COMPACTION_STATE_KEY",
|
||||
"DEFAULT_BACKGROUND_AGENTS_SOURCE_ID",
|
||||
"DEFAULT_FILE_ACCESS_INSTRUCTIONS",
|
||||
"DEFAULT_FILE_ACCESS_SOURCE_ID",
|
||||
"DEFAULT_HARNESS_INSTRUCTIONS",
|
||||
"DEFAULT_MAX_ITERATIONS",
|
||||
"DEFAULT_MEMORY_SOURCE_ID",
|
||||
@@ -334,6 +346,7 @@ __all__ = [
|
||||
"AgentExecutor",
|
||||
"AgentExecutorRequest",
|
||||
"AgentExecutorResponse",
|
||||
"AgentFileStore",
|
||||
"AgentFrameworkException",
|
||||
"AgentMiddleware",
|
||||
"AgentMiddlewareLayer",
|
||||
@@ -393,11 +406,15 @@ __all__ = [
|
||||
"ExperimentalFeature",
|
||||
"FanInEdgeGroup",
|
||||
"FanOutEdgeGroup",
|
||||
"FileAccessProvider",
|
||||
"FileCheckpointStorage",
|
||||
"FileHistoryProvider",
|
||||
"FileSearchMatch",
|
||||
"FileSearchResult",
|
||||
"FileSkill",
|
||||
"FileSkillScript",
|
||||
"FileSkillsSource",
|
||||
"FileSystemAgentFileStore",
|
||||
"FilteringSkillsSource",
|
||||
"FinalT",
|
||||
"FinishReason",
|
||||
@@ -414,6 +431,7 @@ __all__ = [
|
||||
"GeneratedEmbeddings",
|
||||
"GraphConnectivityError",
|
||||
"HistoryProvider",
|
||||
"InMemoryAgentFileStore",
|
||||
"InMemoryCheckpointStorage",
|
||||
"InMemoryHistoryProvider",
|
||||
"InMemorySkillsSource",
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,713 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_framework import (
|
||||
Agent,
|
||||
AgentFileStore,
|
||||
AgentSession,
|
||||
ExperimentalFeature,
|
||||
FileAccessProvider,
|
||||
FileSearchMatch,
|
||||
FileSearchResult,
|
||||
FileSystemAgentFileStore,
|
||||
InMemoryAgentFileStore,
|
||||
Message,
|
||||
SupportsChatGetResponse,
|
||||
)
|
||||
from agent_framework._harness import _file_access as _file_access_module
|
||||
from agent_framework._harness._file_access import (
|
||||
DEFAULT_FILE_ACCESS_INSTRUCTIONS,
|
||||
DEFAULT_FILE_ACCESS_SOURCE_ID,
|
||||
_matches_glob,
|
||||
_normalize_relative_path,
|
||||
_run_search_with_timeout,
|
||||
)
|
||||
|
||||
|
||||
def _tool_by_name(tools: list[object], name: str) -> object:
|
||||
"""Return the tool with the requested name from a prepared tool list."""
|
||||
for tool in tools:
|
||||
if getattr(tool, "name", None) == name:
|
||||
return tool
|
||||
raise AssertionError(f"Tool {name!r} was not found.")
|
||||
|
||||
|
||||
def test_normalize_relative_path_collapses_and_validates() -> None:
|
||||
"""The path normalizer should accept relative forward/backslash paths and reject unsafe ones."""
|
||||
assert _normalize_relative_path("foo/bar.txt") == "foo/bar.txt"
|
||||
assert _normalize_relative_path("foo\\bar.txt") == "foo/bar.txt"
|
||||
assert _normalize_relative_path("foo//bar.txt") == "foo/bar.txt"
|
||||
assert _normalize_relative_path(" foo/bar.txt ") == "foo/bar.txt"
|
||||
assert _normalize_relative_path("", is_directory=True) == ""
|
||||
assert _normalize_relative_path(" ", is_directory=True) == ""
|
||||
assert _normalize_relative_path("sub/", is_directory=True) == "sub"
|
||||
assert _normalize_relative_path("sub\\", is_directory=True) == "sub"
|
||||
|
||||
with pytest.raises(ValueError, match="must not be empty"):
|
||||
_normalize_relative_path("")
|
||||
with pytest.raises(ValueError, match="must not be empty"):
|
||||
_normalize_relative_path(" ")
|
||||
with pytest.raises(ValueError, match="must not end with a path separator"):
|
||||
_normalize_relative_path("foo/")
|
||||
with pytest.raises(ValueError, match="must not end with a path separator"):
|
||||
_normalize_relative_path("foo\\")
|
||||
with pytest.raises(ValueError, match="'..' segments"):
|
||||
_normalize_relative_path("foo/../bar.txt")
|
||||
with pytest.raises(ValueError, match="'..' segments"):
|
||||
_normalize_relative_path("./bar.txt")
|
||||
with pytest.raises(ValueError, match="must be relative"):
|
||||
_normalize_relative_path("C:/abs/path")
|
||||
with pytest.raises(ValueError, match="must be relative"):
|
||||
_normalize_relative_path("\\rooted")
|
||||
with pytest.raises(ValueError, match="must be relative"):
|
||||
_normalize_relative_path("/foo/bar.txt")
|
||||
|
||||
|
||||
def test_matches_glob_is_case_insensitive_and_optional() -> None:
|
||||
"""The glob matcher should be case-insensitive and treat missing patterns as match-all."""
|
||||
assert _matches_glob("notes.MD", "*.md")
|
||||
assert _matches_glob("research_one.txt", "research*")
|
||||
assert not _matches_glob("plan.txt", "*.md")
|
||||
assert _matches_glob("anything", None)
|
||||
assert _matches_glob("anything", "")
|
||||
assert _matches_glob("anything", " ")
|
||||
|
||||
|
||||
def test_file_search_match_round_trips() -> None:
|
||||
"""File search match values should serialize and validate cleanly."""
|
||||
raw_match = {"line_number": 3, "line": "error: boom"}
|
||||
|
||||
match = FileSearchMatch.from_dict(raw_match)
|
||||
assert match == FileSearchMatch(line_number=3, line="error: boom")
|
||||
assert match.to_dict() == raw_match
|
||||
assert "FileSearchMatch(" in repr(match)
|
||||
|
||||
with pytest.raises(ValueError, match="positive integer"):
|
||||
FileSearchMatch(line_number=0, line="oops")
|
||||
with pytest.raises(ValueError, match="must be an integer"):
|
||||
FileSearchMatch.from_dict({"line_number": "1", "line": "oops"})
|
||||
with pytest.raises(ValueError, match="must be a string"):
|
||||
FileSearchMatch.from_dict({"line_number": 1, "line": 42})
|
||||
|
||||
|
||||
def test_file_search_result_round_trips() -> None:
|
||||
"""File search result values should serialize the matching-line list correctly."""
|
||||
raw_result = {
|
||||
"file_name": "notes.md",
|
||||
"snippet": "hello error world",
|
||||
"matching_lines": [{"line_number": 2, "line": "error one"}],
|
||||
}
|
||||
|
||||
result = FileSearchResult.from_dict(raw_result)
|
||||
assert result.file_name == "notes.md"
|
||||
assert result.snippet == "hello error world"
|
||||
assert result.matching_lines == [FileSearchMatch(line_number=2, line="error one")]
|
||||
assert result.to_dict() == raw_result
|
||||
assert json.loads(result.to_json()) == raw_result
|
||||
|
||||
with pytest.raises(ValueError, match="matching_lines must be a list"):
|
||||
FileSearchResult.from_dict({"file_name": "x", "snippet": "", "matching_lines": {}})
|
||||
|
||||
with pytest.raises(ValueError, match="elements must be mappings"):
|
||||
FileSearchResult.from_dict({"file_name": "x", "snippet": "", "matching_lines": ["not-a-dict"]})
|
||||
|
||||
|
||||
async def test_in_memory_store_round_trips_files() -> None:
|
||||
"""The in-memory store should support write/read/exists/delete/list operations."""
|
||||
store = InMemoryAgentFileStore()
|
||||
|
||||
await store.write_file("a.txt", "alpha")
|
||||
await store.write_file("sub/b.txt", "beta")
|
||||
|
||||
assert await store.file_exists("a.txt")
|
||||
assert not await store.file_exists("missing.txt")
|
||||
assert await store.read_file("a.txt") == "alpha"
|
||||
assert await store.read_file("missing.txt") is None
|
||||
|
||||
assert sorted(await store.list_files()) == ["a.txt"] # subdirs are not direct children
|
||||
assert sorted(await store.list_files("sub")) == ["b.txt"]
|
||||
|
||||
assert await store.delete_file("a.txt") is True
|
||||
assert await store.delete_file("a.txt") is False
|
||||
assert sorted(await store.list_files()) == []
|
||||
|
||||
|
||||
async def test_in_memory_store_search_returns_matches_with_snippets() -> None:
|
||||
"""The in-memory store should search file content case-insensitively and respect glob filters."""
|
||||
store = InMemoryAgentFileStore()
|
||||
await store.write_file("a.md", "line one\nThis line has ERROR inside\nline three\r")
|
||||
await store.write_file("b.md", "no match here")
|
||||
await store.write_file("notes.txt", "ERROR but wrong extension")
|
||||
|
||||
results = await store.search_files("", "error", "*.md")
|
||||
assert [result.file_name for result in results] == ["a.md"]
|
||||
matching_lines = results[0].matching_lines
|
||||
assert matching_lines == [FileSearchMatch(line_number=2, line="This line has ERROR inside")]
|
||||
assert "ERROR" in results[0].snippet
|
||||
|
||||
# No glob -> searches every file.
|
||||
results_all = await store.search_files("", "error")
|
||||
assert {result.file_name for result in results_all} == {"a.md", "notes.txt"}
|
||||
|
||||
|
||||
async def test_in_memory_store_search_rejects_invalid_and_oversize_regex() -> None:
|
||||
"""``search_files`` should surface clean errors for bad regex input."""
|
||||
store = InMemoryAgentFileStore()
|
||||
await store.write_file("a.md", "hello")
|
||||
|
||||
with pytest.raises(re.error):
|
||||
await store.search_files("", "[unclosed")
|
||||
|
||||
with pytest.raises(ValueError, match="too long"):
|
||||
await store.search_files("", "a" * 257)
|
||||
|
||||
|
||||
async def test_in_memory_store_normalizes_paths() -> None:
|
||||
"""Path normalization should reject traversal in the in-memory store too."""
|
||||
store = InMemoryAgentFileStore()
|
||||
for bad in ("../escape.txt", "/abs/path.txt", "."):
|
||||
with pytest.raises(ValueError):
|
||||
await store.write_file(bad, "boom")
|
||||
|
||||
|
||||
async def test_filesystem_store_round_trips_files(tmp_path: Path) -> None:
|
||||
"""The filesystem store should round-trip files on disk and create parents on write."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
|
||||
await store.write_file("nested/a.txt", "alpha")
|
||||
assert (tmp_path / "nested" / "a.txt").read_text(encoding="utf-8") == "alpha"
|
||||
|
||||
assert await store.read_file("nested/a.txt") == "alpha"
|
||||
assert await store.read_file("missing.txt") is None
|
||||
assert await store.file_exists("nested/a.txt")
|
||||
assert not await store.file_exists("missing.txt")
|
||||
assert sorted(await store.list_files("nested")) == ["a.txt"]
|
||||
assert sorted(await store.list_files()) == [] # root only contains the directory
|
||||
|
||||
assert await store.delete_file("nested/a.txt") is True
|
||||
assert await store.delete_file("nested/a.txt") is False
|
||||
|
||||
|
||||
async def test_filesystem_store_rejects_traversal_and_rooted_paths(tmp_path: Path) -> None:
|
||||
"""The filesystem store should refuse paths that escape the configured root."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
|
||||
for bad in ("../escape.txt", "/etc/passwd", "C:/Windows/System32/notepad.exe", ".", ".."):
|
||||
with pytest.raises(ValueError):
|
||||
await store.write_file(bad, "boom")
|
||||
|
||||
|
||||
async def test_filesystem_store_rejects_symlinks_into_root(tmp_path: Path) -> None:
|
||||
"""The filesystem store should refuse to read through a symlink target."""
|
||||
target = tmp_path / "outside.txt"
|
||||
target.write_text("outside", encoding="utf-8")
|
||||
root = tmp_path / "root"
|
||||
root.mkdir()
|
||||
link = root / "link.txt"
|
||||
try:
|
||||
link.symlink_to(target)
|
||||
except (OSError, NotImplementedError) as exc:
|
||||
pytest.skip(f"Symbolic links are not supported in this environment: {exc!r}")
|
||||
|
||||
store = FileSystemAgentFileStore(root)
|
||||
with pytest.raises(ValueError, match="symbolic link"):
|
||||
await store.read_file("link.txt")
|
||||
with pytest.raises(ValueError, match="symbolic link"):
|
||||
await store.write_file("link.txt", "stomp")
|
||||
|
||||
# List operations should silently skip the symlink entry rather than raise.
|
||||
assert await store.list_files() == []
|
||||
|
||||
|
||||
async def test_filesystem_store_rejects_in_root_symlinks(tmp_path: Path) -> None:
|
||||
"""Symlinks whose target lives under the root must still be rejected.
|
||||
|
||||
``Path.resolve`` collapses the symlink, so a naive resolved-path check
|
||||
would silently follow it. The symlink probe must operate on the
|
||||
unresolved candidate for this case to fail closed.
|
||||
"""
|
||||
root = tmp_path / "root"
|
||||
root.mkdir()
|
||||
real = root / "real.txt"
|
||||
real.write_text("payload", encoding="utf-8")
|
||||
link = root / "alias.txt"
|
||||
try:
|
||||
link.symlink_to(real)
|
||||
except (OSError, NotImplementedError) as exc:
|
||||
pytest.skip(f"Symbolic links are not supported in this environment: {exc!r}")
|
||||
|
||||
store = FileSystemAgentFileStore(root)
|
||||
with pytest.raises(ValueError, match="symbolic link"):
|
||||
await store.read_file("alias.txt")
|
||||
# The non-symlinked sibling must still be readable.
|
||||
assert await store.read_file("real.txt") == "payload"
|
||||
|
||||
|
||||
async def test_filesystem_store_search_matches_lines_and_filters_globs(tmp_path: Path) -> None:
|
||||
"""The filesystem store should search files on disk and apply glob filters by file name."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
await store.write_file("a.md", "hello\nERROR happens\nbye\r")
|
||||
await store.write_file("b.txt", "ERROR happens too")
|
||||
await store.write_file("c.md", "nothing here")
|
||||
|
||||
results = await store.search_files("", "error", "*.md")
|
||||
assert [result.file_name for result in results] == ["a.md"]
|
||||
assert results[0].matching_lines == [FileSearchMatch(line_number=2, line="ERROR happens")]
|
||||
assert "ERROR" in results[0].snippet
|
||||
|
||||
results_all = await store.search_files("", "error")
|
||||
assert {result.file_name for result in results_all} == {"a.md", "b.txt"}
|
||||
|
||||
|
||||
async def test_filesystem_store_search_skips_non_utf8_files(tmp_path: Path) -> None:
|
||||
"""The filesystem store should silently skip non-UTF-8 files instead of aborting the search."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
await store.write_file("notes.md", "ERROR happens here")
|
||||
(tmp_path / "blob.bin").write_bytes(b"\x80\x81\x82\x83")
|
||||
|
||||
results = await store.search_files("", "error")
|
||||
assert [result.file_name for result in results] == ["notes.md"]
|
||||
|
||||
|
||||
async def test_filesystem_store_create_directory(tmp_path: Path) -> None:
|
||||
"""The filesystem store should create directories under the configured root."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
await store.create_directory("nested/inner")
|
||||
assert (tmp_path / "nested" / "inner").is_dir()
|
||||
|
||||
|
||||
async def test_filesystem_store_list_files_accepts_blank_directory(tmp_path: Path) -> None:
|
||||
"""Whitespace-only directory inputs should resolve to the root, matching the in-memory store."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
await store.write_file("a.txt", "alpha")
|
||||
assert sorted(await store.list_files("")) == ["a.txt"]
|
||||
assert sorted(await store.list_files(" ")) == ["a.txt"]
|
||||
|
||||
|
||||
def test_filesystem_store_requires_non_empty_root() -> None:
|
||||
"""The filesystem store constructor should refuse blank root paths."""
|
||||
with pytest.raises(ValueError, match="must not be empty"):
|
||||
FileSystemAgentFileStore("")
|
||||
with pytest.raises(ValueError, match="must not be empty"):
|
||||
FileSystemAgentFileStore(" ")
|
||||
|
||||
|
||||
async def test_file_access_provider_registers_tools_and_instructions(
|
||||
chat_client_base: SupportsChatGetResponse,
|
||||
) -> None:
|
||||
"""``FileAccessProvider.before_run`` should add the canonical instructions and five tools."""
|
||||
session = AgentSession(session_id="session-1")
|
||||
store = InMemoryAgentFileStore()
|
||||
provider = FileAccessProvider(store=store)
|
||||
agent = Agent(client=chat_client_base, context_providers=[provider])
|
||||
|
||||
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
|
||||
session=session,
|
||||
input_messages=[Message(role="user", contents=["work with files"])],
|
||||
)
|
||||
|
||||
tools = options["tools"]
|
||||
assert isinstance(tools, list)
|
||||
expected_names = {
|
||||
"file_access_save_file",
|
||||
"file_access_read_file",
|
||||
"file_access_delete_file",
|
||||
"file_access_list_files",
|
||||
"file_access_search_files",
|
||||
}
|
||||
assert {getattr(tool, "name", None) for tool in tools} >= expected_names
|
||||
|
||||
instructions = options.get("instructions")
|
||||
if isinstance(instructions, str):
|
||||
assert DEFAULT_FILE_ACCESS_INSTRUCTIONS in instructions
|
||||
else:
|
||||
assert any(DEFAULT_FILE_ACCESS_INSTRUCTIONS in chunk for chunk in (instructions or []))
|
||||
|
||||
|
||||
async def test_file_access_provider_delete_approval_defaults_to_always_require(
|
||||
chat_client_base: SupportsChatGetResponse,
|
||||
) -> None:
|
||||
"""By default ``file_access_delete_file`` should require host approval."""
|
||||
session = AgentSession(session_id="session-1")
|
||||
provider = FileAccessProvider(store=InMemoryAgentFileStore())
|
||||
agent = Agent(client=chat_client_base, context_providers=[provider])
|
||||
|
||||
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
|
||||
session=session,
|
||||
input_messages=[Message(role="user", contents=["work with files"])],
|
||||
)
|
||||
|
||||
tools = options["tools"]
|
||||
assert isinstance(tools, list)
|
||||
delete_file = _tool_by_name(tools, "file_access_delete_file")
|
||||
assert delete_file.approval_mode == "always_require"
|
||||
# The non-destructive tools should remain autonomous.
|
||||
for name in (
|
||||
"file_access_save_file",
|
||||
"file_access_read_file",
|
||||
"file_access_list_files",
|
||||
"file_access_search_files",
|
||||
):
|
||||
assert _tool_by_name(tools, name).approval_mode == "never_require"
|
||||
|
||||
|
||||
async def test_file_access_provider_delete_approval_opt_out(
|
||||
chat_client_base: SupportsChatGetResponse,
|
||||
) -> None:
|
||||
"""``require_delete_approval=False`` should drop delete to ``never_require``."""
|
||||
session = AgentSession(session_id="session-1")
|
||||
provider = FileAccessProvider(store=InMemoryAgentFileStore(), require_delete_approval=False)
|
||||
agent = Agent(client=chat_client_base, context_providers=[provider])
|
||||
|
||||
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
|
||||
session=session,
|
||||
input_messages=[Message(role="user", contents=["work with files"])],
|
||||
)
|
||||
|
||||
delete_file = _tool_by_name(options["tools"], "file_access_delete_file") # type: ignore[arg-type]
|
||||
assert delete_file.approval_mode == "never_require"
|
||||
|
||||
|
||||
async def test_file_access_provider_tools_round_trip_files(
|
||||
chat_client_base: SupportsChatGetResponse,
|
||||
) -> None:
|
||||
"""The provider's tools should drive save/read/list/search/delete flows on an ``InMemoryAgentFileStore``."""
|
||||
session = AgentSession(session_id="session-1")
|
||||
store = InMemoryAgentFileStore()
|
||||
provider = FileAccessProvider(store=store)
|
||||
agent = Agent(client=chat_client_base, context_providers=[provider])
|
||||
|
||||
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
|
||||
session=session,
|
||||
input_messages=[Message(role="user", contents=["work with files"])],
|
||||
)
|
||||
tools = options["tools"]
|
||||
assert isinstance(tools, list)
|
||||
|
||||
save_file = _tool_by_name(tools, "file_access_save_file")
|
||||
read_file = _tool_by_name(tools, "file_access_read_file")
|
||||
delete_file = _tool_by_name(tools, "file_access_delete_file")
|
||||
list_files = _tool_by_name(tools, "file_access_list_files")
|
||||
search_files = _tool_by_name(tools, "file_access_search_files")
|
||||
|
||||
saved = await save_file.invoke(arguments={"file_name": "plan.md", "content": "step 1\nERROR step 2"})
|
||||
assert "plan.md" in saved[0].text and "saved" in saved[0].text
|
||||
|
||||
# Default overwrite=False should refuse the second save.
|
||||
refused = await save_file.invoke(arguments={"file_name": "plan.md", "content": "stomp"})
|
||||
assert "already exists" in refused[0].text
|
||||
|
||||
# overwrite=True should succeed.
|
||||
overwritten = await save_file.invoke(
|
||||
arguments={"file_name": "plan.md", "content": "stomp\nERROR replaced", "overwrite": True}
|
||||
)
|
||||
assert "saved" in overwritten[0].text
|
||||
|
||||
read_back = await read_file.invoke(arguments={"file_name": "plan.md"})
|
||||
assert read_back[0].text == "stomp\nERROR replaced"
|
||||
|
||||
listed = await list_files.invoke()
|
||||
assert json.loads(listed[0].text) == ["plan.md"]
|
||||
|
||||
# The list tool should accept an optional directory argument so agents can
|
||||
# enumerate nested folders (not only the root).
|
||||
await save_file.invoke(arguments={"file_name": "reports/2024.md", "content": "annual"})
|
||||
listed_nested = await list_files.invoke(arguments={"directory": "reports"})
|
||||
assert json.loads(listed_nested[0].text) == ["2024.md"]
|
||||
# Blank / whitespace directory should fall back to the root listing.
|
||||
listed_blank = await list_files.invoke(arguments={"directory": " "})
|
||||
assert sorted(json.loads(listed_blank[0].text)) == ["plan.md"]
|
||||
|
||||
missing = await read_file.invoke(arguments={"file_name": "missing.md"})
|
||||
assert "not found" in missing[0].text
|
||||
|
||||
search_payload = await search_files.invoke(arguments={"regex_pattern": "error", "file_pattern": "*.md"})
|
||||
parsed = json.loads(search_payload[0].text)
|
||||
assert parsed[0]["file_name"] == "plan.md"
|
||||
assert parsed[0]["matching_lines"][0]["line"] == "ERROR replaced"
|
||||
|
||||
# The search tool should likewise accept an optional directory argument so
|
||||
# agents can scope a search to a subfolder.
|
||||
await save_file.invoke(arguments={"file_name": "reports/issues.md", "content": "ERROR nested"})
|
||||
scoped = await search_files.invoke(
|
||||
arguments={"regex_pattern": "error", "file_pattern": "*.md", "directory": "reports"}
|
||||
)
|
||||
scoped_parsed = json.loads(scoped[0].text)
|
||||
assert [entry["file_name"] for entry in scoped_parsed] == ["issues.md"]
|
||||
|
||||
deleted = await delete_file.invoke(arguments={"file_name": "plan.md"})
|
||||
assert "deleted" in deleted[0].text
|
||||
|
||||
missing_delete = await delete_file.invoke(arguments={"file_name": "plan.md"})
|
||||
assert "not found" in missing_delete[0].text
|
||||
|
||||
|
||||
async def test_file_access_provider_accepts_custom_instructions() -> None:
|
||||
"""Custom instructions should override the default banner."""
|
||||
store = InMemoryAgentFileStore()
|
||||
provider = FileAccessProvider(store=store, instructions="custom-banner")
|
||||
assert provider.instructions == "custom-banner"
|
||||
assert provider.source_id == DEFAULT_FILE_ACCESS_SOURCE_ID
|
||||
|
||||
|
||||
async def test_in_memory_store_write_file_raises_when_exists_and_no_overwrite() -> None:
|
||||
"""The atomic exclusive-create path should raise ``FileExistsError`` under the lock."""
|
||||
store = InMemoryAgentFileStore()
|
||||
await store.write_file("plan.md", "v1")
|
||||
|
||||
with pytest.raises(FileExistsError):
|
||||
await store.write_file("plan.md", "v2", overwrite=False)
|
||||
|
||||
# The original content is preserved.
|
||||
assert await store.read_file("plan.md") == "v1"
|
||||
|
||||
# Default ``overwrite=True`` still replaces.
|
||||
await store.write_file("plan.md", "v3")
|
||||
assert await store.read_file("plan.md") == "v3"
|
||||
|
||||
|
||||
async def test_filesystem_store_write_file_raises_when_exists_and_no_overwrite(tmp_path: Path) -> None:
|
||||
"""The filesystem store should use exclusive-create semantics when ``overwrite=False``."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
await store.write_file("plan.md", "v1")
|
||||
|
||||
with pytest.raises(FileExistsError):
|
||||
await store.write_file("plan.md", "v2", overwrite=False)
|
||||
|
||||
assert (tmp_path / "plan.md").read_text(encoding="utf-8") == "v1"
|
||||
|
||||
await store.write_file("plan.md", "v3", overwrite=True)
|
||||
assert (tmp_path / "plan.md").read_text(encoding="utf-8") == "v3"
|
||||
|
||||
|
||||
async def test_run_search_with_timeout_raises_value_error(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A scan that exceeds the timeout should surface a clean ``ValueError``."""
|
||||
monkeypatch.setattr(_file_access_module, "_SEARCH_TIMEOUT_SECONDS", 0.01)
|
||||
|
||||
def slow() -> list[FileSearchResult]:
|
||||
time.sleep(0.5)
|
||||
return []
|
||||
|
||||
with pytest.raises(ValueError, match="did not complete"):
|
||||
await _run_search_with_timeout(slow)
|
||||
|
||||
|
||||
async def test_filesystem_store_symlink_probe_fails_closed_on_oserror(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""If ``Path.is_symlink`` raises during the probe, the operation must be refused."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
await store.write_file("ok.txt", "content")
|
||||
|
||||
def boom(self: Path) -> bool:
|
||||
raise PermissionError("access denied")
|
||||
|
||||
monkeypatch.setattr(Path, "is_symlink", boom)
|
||||
|
||||
with pytest.raises(ValueError, match="symbolic link or reparse point"):
|
||||
await store.read_file("ok.txt")
|
||||
|
||||
|
||||
def test_file_access_harness_classes_are_marked_experimental() -> None:
|
||||
"""File-access harness public classes should expose HARNESS experimental metadata."""
|
||||
assert AgentFileStore.__feature_id__ == ExperimentalFeature.HARNESS.value
|
||||
assert InMemoryAgentFileStore.__feature_id__ == ExperimentalFeature.HARNESS.value
|
||||
assert FileSystemAgentFileStore.__feature_id__ == ExperimentalFeature.HARNESS.value
|
||||
assert FileSearchMatch.__feature_id__ == ExperimentalFeature.HARNESS.value
|
||||
assert FileSearchResult.__feature_id__ == ExperimentalFeature.HARNESS.value
|
||||
assert FileAccessProvider.__feature_id__ == ExperimentalFeature.HARNESS.value
|
||||
assert ".. warning:: Experimental" in (FileAccessProvider.__doc__ or "")
|
||||
|
||||
|
||||
async def test_in_memory_store_preserves_original_case_on_list_and_search() -> None:
|
||||
"""``list_files`` / ``search_files`` should return original-case names, not lowercased keys.
|
||||
|
||||
Matches :class:`FileSystemAgentFileStore` on case-preserving filesystems so
|
||||
tests written against the in-memory backend cannot encode a contract that
|
||||
will diverge in production.
|
||||
"""
|
||||
store = InMemoryAgentFileStore()
|
||||
await store.write_file("Plan.MD", "ERROR happens here\n")
|
||||
await store.write_file("Reports/Q1.MD", "alpha")
|
||||
|
||||
# list_files keeps the original case
|
||||
assert sorted(await store.list_files()) == ["Plan.MD"]
|
||||
assert sorted(await store.list_files("Reports")) == ["Q1.MD"]
|
||||
|
||||
# case-insensitive directory lookup still works
|
||||
assert sorted(await store.list_files("reports")) == ["Q1.MD"]
|
||||
|
||||
# search_files emits the original-case file name in FileSearchResult
|
||||
results = await store.search_files("", "error", "*.MD")
|
||||
assert [r.file_name for r in results] == ["Plan.MD"]
|
||||
|
||||
# read_file remains case-insensitive
|
||||
assert await store.read_file("plan.md") == "ERROR happens here\n"
|
||||
|
||||
|
||||
async def test_filesystem_store_read_file_raises_value_error_on_non_utf8(tmp_path: Path) -> None:
|
||||
"""Binary / non-UTF-8 files should raise a clean ``ValueError`` rather than ``UnicodeDecodeError``.
|
||||
|
||||
The tool-layer wrapper relies on this contract to convert the failure into
|
||||
a recoverable string response for the agent.
|
||||
"""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
(tmp_path / "blob.bin").write_bytes(b"\x80\x81\x82\x83")
|
||||
|
||||
with pytest.raises(ValueError, match="not UTF-8 text"):
|
||||
await store.read_file("blob.bin")
|
||||
|
||||
|
||||
async def test_filesystem_store_search_logs_skipped_non_utf8_files(
|
||||
tmp_path: Path, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
"""``search_files`` skips non-UTF-8 files but logs per-file and a summary so operators have signal."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
await store.write_file("notes.md", "ERROR happens here")
|
||||
(tmp_path / "blob.bin").write_bytes(b"\x80\x81\x82\x83")
|
||||
|
||||
with caplog.at_level("INFO", logger="agent_framework._harness._file_access"):
|
||||
results = await store.search_files("", "error")
|
||||
|
||||
assert [r.file_name for r in results] == ["notes.md"]
|
||||
assert any("Skipping non-UTF-8 file during search" in rec.message for rec in caplog.records)
|
||||
assert any("skipped 1 non-UTF-8 file" in rec.message for rec in caplog.records)
|
||||
|
||||
|
||||
async def test_file_access_tool_wrappers_surface_value_error_as_message(
|
||||
chat_client_base: SupportsChatGetResponse,
|
||||
) -> None:
|
||||
"""Recoverable failures (bad path, oversized regex, non-UTF-8 read) should be returned as strings.
|
||||
|
||||
Without these wrappers the model sees a raw stack trace for "you used ``..``"
|
||||
but a polite message for "the file already exists", which is the opposite
|
||||
of what is recoverable.
|
||||
"""
|
||||
session = AgentSession(session_id="session-1")
|
||||
store = InMemoryAgentFileStore()
|
||||
provider = FileAccessProvider(store=store)
|
||||
agent = Agent(client=chat_client_base, context_providers=[provider])
|
||||
|
||||
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
|
||||
session=session,
|
||||
input_messages=[Message(role="user", contents=["work with files"])],
|
||||
)
|
||||
tools = options["tools"]
|
||||
assert isinstance(tools, list)
|
||||
|
||||
save_file = _tool_by_name(tools, "file_access_save_file")
|
||||
read_file = _tool_by_name(tools, "file_access_read_file")
|
||||
delete_file = _tool_by_name(tools, "file_access_delete_file")
|
||||
list_files = _tool_by_name(tools, "file_access_list_files")
|
||||
search_files = _tool_by_name(tools, "file_access_search_files")
|
||||
|
||||
# Path-traversal attempts on each tool should return a clean string, not raise.
|
||||
saved = await save_file.invoke(arguments={"file_name": "../escape.txt", "content": "x"})
|
||||
assert "Could not save" in saved[0].text and "escape" in saved[0].text.lower()
|
||||
read = await read_file.invoke(arguments={"file_name": "../escape.txt"})
|
||||
assert "Could not read" in read[0].text
|
||||
deleted = await delete_file.invoke(arguments={"file_name": "../escape.txt"})
|
||||
assert "Could not delete" in deleted[0].text
|
||||
listed = await list_files.invoke(arguments={"directory": "../escape"})
|
||||
assert "Could not list" in listed[0].text
|
||||
|
||||
# Regex length cap should also be returned to the model as text.
|
||||
too_long = "a" * 1024
|
||||
searched = await search_files.invoke(arguments={"regex_pattern": too_long})
|
||||
assert "Could not search files" in searched[0].text
|
||||
|
||||
|
||||
async def test_file_access_tool_read_file_wrapper_surfaces_non_utf8(
|
||||
tmp_path: Path, chat_client_base: SupportsChatGetResponse
|
||||
) -> None:
|
||||
"""The read-file tool wrapper should convert a non-UTF-8 ``ValueError`` into a readable string."""
|
||||
store = FileSystemAgentFileStore(tmp_path)
|
||||
(tmp_path / "blob.bin").write_bytes(b"\x80\x81\x82\x83")
|
||||
|
||||
session = AgentSession(session_id="session-1")
|
||||
provider = FileAccessProvider(store=store)
|
||||
agent = Agent(client=chat_client_base, context_providers=[provider])
|
||||
|
||||
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
|
||||
session=session,
|
||||
input_messages=[Message(role="user", contents=["read it"])],
|
||||
)
|
||||
read_file = _tool_by_name(options["tools"], "file_access_read_file")
|
||||
response = await read_file.invoke(arguments={"file_name": "blob.bin"})
|
||||
assert "Could not read" in response[0].text and "UTF-8" in response[0].text
|
||||
|
||||
|
||||
_NEEDS_SYMLINK = "Symbolic links are not supported in this environment"
|
||||
|
||||
|
||||
async def test_filesystem_store_rejects_symlink_on_delete_search_and_list(tmp_path: Path) -> None:
|
||||
"""The same symlink probe must front delete/search/list, not just read/write."""
|
||||
target = tmp_path / "outside.txt"
|
||||
target.write_text("outside", encoding="utf-8")
|
||||
root = tmp_path / "root"
|
||||
root.mkdir()
|
||||
link = root / "link.txt"
|
||||
try:
|
||||
link.symlink_to(target)
|
||||
except (OSError, NotImplementedError) as exc:
|
||||
pytest.skip(f"{_NEEDS_SYMLINK}: {exc!r}")
|
||||
|
||||
store = FileSystemAgentFileStore(root)
|
||||
|
||||
with pytest.raises(ValueError, match="symbolic link"):
|
||||
await store.delete_file("link.txt")
|
||||
|
||||
# search_files of the root never touches the symlink leaf directly, but
|
||||
# search_files of a symlinked *directory* path must be rejected by the
|
||||
# safe-directory resolver.
|
||||
dir_link = root / "alias_dir"
|
||||
other_dir = tmp_path / "outside_dir"
|
||||
other_dir.mkdir()
|
||||
try:
|
||||
dir_link.symlink_to(other_dir)
|
||||
except (OSError, NotImplementedError) as exc:
|
||||
pytest.skip(f"{_NEEDS_SYMLINK}: {exc!r}")
|
||||
|
||||
with pytest.raises(ValueError, match="symbolic link"):
|
||||
await store.search_files("alias_dir", "anything")
|
||||
with pytest.raises(ValueError, match="symbolic link"):
|
||||
await store.list_files("alias_dir")
|
||||
|
||||
|
||||
async def test_filesystem_store_rejects_symlinked_intermediate_directory(tmp_path: Path) -> None:
|
||||
"""A symlink used as a non-leaf path segment must still be rejected.
|
||||
|
||||
The classic escape vector is ``root/aliased_dir/file.txt`` where
|
||||
``aliased_dir`` is a symlink to somewhere outside the root. The
|
||||
``_throw_if_contains_symlink`` walk must check every segment, not only
|
||||
the leaf.
|
||||
"""
|
||||
outside = tmp_path / "outside_dir"
|
||||
outside.mkdir()
|
||||
(outside / "secret.txt").write_text("payload", encoding="utf-8")
|
||||
root = tmp_path / "root"
|
||||
root.mkdir()
|
||||
link = root / "aliased_dir"
|
||||
try:
|
||||
link.symlink_to(outside)
|
||||
except (OSError, NotImplementedError) as exc:
|
||||
pytest.skip(f"{_NEEDS_SYMLINK}: {exc!r}")
|
||||
|
||||
store = FileSystemAgentFileStore(root)
|
||||
|
||||
for op in ("read", "write", "delete"):
|
||||
with pytest.raises(ValueError, match="symbolic link"):
|
||||
if op == "read":
|
||||
await store.read_file("aliased_dir/secret.txt")
|
||||
elif op == "write":
|
||||
await store.write_file("aliased_dir/secret.txt", "stomp")
|
||||
else:
|
||||
await store.delete_file("aliased_dir/secret.txt")
|
||||
@@ -8,6 +8,7 @@ These samples demonstrate how to use context providers to enrich agent conversat
|
||||
|---------------|-------------|
|
||||
| [`simple_context_provider.py`](simple_context_provider.py) | Implement a custom context provider by extending `ContextProvider` to extract and inject structured user information across turns. |
|
||||
| [`azure_ai_foundry_memory.py`](azure_ai_foundry_memory.py) | Use `FoundryMemoryProvider` to add semantic memory — automatically retrieves, searches, and stores memories via Azure AI Foundry. |
|
||||
| [`file_access_data_processing/`](file_access_data_processing/) | Use `FileAccessProvider` with `FileSystemAgentFileStore` to give an agent read/write/search access to a folder of CSV data files. See its own [README](file_access_data_processing/README.md). |
|
||||
| [`azure_ai_search/`](azure_ai_search/) | Retrieval Augmented Generation (RAG) with Azure AI Search in semantic and agentic modes. See its own [README](azure_ai_search/README.md). |
|
||||
| [`mem0/`](mem0/) | Memory-powered context using the Mem0 integration (open-source and managed). See its own [README](mem0/README.md). |
|
||||
| [`redis/`](redis/) | Redis-backed context providers for conversation memory and sessions. See its own [README](redis/README.md). |
|
||||
@@ -25,4 +26,9 @@ These samples demonstrate how to use context providers to enrich agent conversat
|
||||
- `AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME`: Embedding model deployment name (e.g., `text-embedding-ada-002`)
|
||||
- Azure CLI authentication (`az login`)
|
||||
|
||||
**For `file_access_data_processing/`:**
|
||||
- `FOUNDRY_PROJECT_ENDPOINT`: Your Azure AI Foundry project endpoint
|
||||
- `FOUNDRY_MODEL`: Chat model deployment name
|
||||
- Azure CLI authentication (`az login`)
|
||||
|
||||
See each subfolder's README for provider-specific prerequisites.
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
# File Access Data Processing
|
||||
|
||||
This sample demonstrates how to give an `Agent` access to a folder of data files
|
||||
by attaching `FileAccessProvider` (backed by `FileSystemAgentFileStore`) as a
|
||||
context provider.
|
||||
|
||||
The agent is given a `working/` folder containing `sales.csv` — ~50 rows of
|
||||
sales transaction data — and is driven through a short scripted conversation
|
||||
that exercises every tool the provider exposes:
|
||||
|
||||
| Step | Prompt | Tool(s) used |
|
||||
|---|---|---|
|
||||
| 1 | "What files do you have access to?" | `file_access_list_files` |
|
||||
| 2 | "Read sales.csv and summarize…" | `file_access_read_file` |
|
||||
| 3 | "Calculate the total revenue per region…" | (uses previously read data) |
|
||||
| 4 | "Save a markdown report named `region_totals.md`…" | `file_access_save_file` |
|
||||
| 5 | "List the files again so I can confirm…" | `file_access_list_files` |
|
||||
|
||||
After the run, the sample prints the final contents of `working/` so the
|
||||
written file is easy to spot.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `FOUNDRY_PROJECT_ENDPOINT` | Your Azure AI Foundry project endpoint. |
|
||||
| `FOUNDRY_MODEL` | Chat model deployment name (e.g. `gpt-4o`). |
|
||||
|
||||
Run `az login` before executing the sample so `AzureCliCredential` can
|
||||
authenticate.
|
||||
|
||||
## Running the sample
|
||||
|
||||
From `python/`:
|
||||
|
||||
```bash
|
||||
uv run --package agent-framework-core python samples/02-agents/context_providers/file_access_data_processing/data_processing.py
|
||||
```
|
||||
|
||||
Or directly:
|
||||
|
||||
```bash
|
||||
python samples/02-agents/context_providers/file_access_data_processing/data_processing.py
|
||||
```
|
||||
|
||||
## Sample data
|
||||
|
||||
`working/sales.csv` contains January–March 2025 sales transactions with these
|
||||
columns:
|
||||
|
||||
| Column | Description |
|
||||
|---|---|
|
||||
| `date` | Transaction date (YYYY-MM-DD) |
|
||||
| `product` | Product name |
|
||||
| `category` | Product category (Electronics, Furniture, Stationery) |
|
||||
| `quantity` | Units sold |
|
||||
| `unit_price` | Price per unit |
|
||||
| `region` | Sales region (North, South, West) |
|
||||
| `salesperson` | Name of the salesperson |
|
||||
|
||||
The sample writes `region_totals.md` into the same folder. Delete it between
|
||||
runs if you want a clean state.
|
||||
+145
@@ -0,0 +1,145 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Sample: use ``FileAccessProvider`` to give an agent access to a folder of CSV data files.
|
||||
|
||||
This sample demonstrates how to attach :class:`FileAccessProvider` (backed by
|
||||
:class:`FileSystemAgentFileStore`) to an ``Agent`` so the model can read input
|
||||
data, perform analysis, and write summary output back to the same folder via
|
||||
the ``file_access_*`` tools.
|
||||
|
||||
The sibling ``working/`` folder contains ``sales.csv`` — ~50 rows of sales
|
||||
transactions (date, product, category, quantity, unit_price, region,
|
||||
salesperson). The agent is asked, in a single session, to: list available
|
||||
files, inspect the data, compute regional totals, and save a markdown summary.
|
||||
|
||||
Prerequisites:
|
||||
- ``FOUNDRY_PROJECT_ENDPOINT``: Your Azure AI Foundry project endpoint.
|
||||
- ``FOUNDRY_MODEL``: Chat model deployment name.
|
||||
- Run ``az login`` before executing the sample.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from agent_framework import Agent, FileAccessProvider, FileSystemAgentFileStore
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from azure.identity import AzureCliCredential
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load python/.env (python-dotenv walks up from this file by default). Pass
|
||||
# override=True so values from .env take precedence over any pre-existing OS
|
||||
# environment variables — without this, OS-level values silently win.
|
||||
load_dotenv(override=True)
|
||||
|
||||
INSTRUCTIONS = """
|
||||
You are a data analyst assistant. You have access to a folder of data files via
|
||||
the file_access_* tools.
|
||||
|
||||
## Getting started
|
||||
- Start by listing available files with file_access_list_files to see what data
|
||||
is available.
|
||||
- Read the files to understand their structure and contents.
|
||||
|
||||
## Working with data
|
||||
- When asked to analyze data, read the relevant files first, then perform the
|
||||
analysis.
|
||||
- Show your analysis clearly with tables, summaries, and key insights.
|
||||
- When calculations are needed, work through them step by step and show your
|
||||
reasoning.
|
||||
|
||||
## Writing output
|
||||
- When asked to produce output files (e.g., reports, summaries, filtered data),
|
||||
use file_access_save_file to write them.
|
||||
- Use appropriate file formats: CSV for tabular data, Markdown for reports.
|
||||
- Confirm what you wrote and where.
|
||||
|
||||
## Important
|
||||
- Never modify or delete the original input data files unless explicitly asked
|
||||
to do so.
|
||||
- If asked about data you haven't read yet, read it first before answering.
|
||||
- Always explain your reasoning between tool calls so the user can follow along.
|
||||
"""
|
||||
|
||||
PROMPTS = [
|
||||
"What files do you have access to?",
|
||||
"Read sales.csv and summarize what columns it contains and how many rows it has.",
|
||||
"Calculate the total revenue (quantity * unit_price) per region and show the result as a table.",
|
||||
(
|
||||
"Save a markdown report named region_totals.md that contains the regional totals "
|
||||
"and a one-paragraph summary of which region performed best."
|
||||
),
|
||||
"List the files again so I can confirm region_totals.md was created.",
|
||||
]
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
# 1. Resolve the working directory bundled alongside this script.
|
||||
working_dir = Path(__file__).parent / "working"
|
||||
|
||||
# 2. Build the chat client.
|
||||
client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=AzureCliCredential(),
|
||||
)
|
||||
|
||||
# 3. Wire up the file access provider against a file-system-backed store
|
||||
# rooted at the sample's working/ folder. The provider injects its
|
||||
# default instructions plus exposes five file_access_* tools to the
|
||||
# agent for the duration of each run.
|
||||
file_access = FileAccessProvider(store=FileSystemAgentFileStore(working_dir))
|
||||
|
||||
# 4. Create the agent and attach the provider.
|
||||
async with Agent(
|
||||
client=client,
|
||||
name="DataAnalyst",
|
||||
description="A data analyst assistant that reads, analyzes, and processes data files.",
|
||||
instructions=INSTRUCTIONS,
|
||||
context_providers=[file_access],
|
||||
) as agent:
|
||||
# 5. Run all prompts inside one session so the conversation remains
|
||||
# coherent across turns.
|
||||
session = agent.create_session()
|
||||
for prompt in PROMPTS:
|
||||
print(f"\nUser: {prompt}")
|
||||
response = await agent.run(prompt, session=session)
|
||||
print(f"Assistant: {response}")
|
||||
|
||||
# 6. Show the final folder contents so the side effects of the run are
|
||||
# visible to the reader.
|
||||
print("\nFinal contents of working/:")
|
||||
for path in sorted(working_dir.iterdir()):
|
||||
print(f" - {path.name} ({path.stat().st_size} bytes)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
|
||||
# Sample output (truncated):
|
||||
#
|
||||
# User: What files do you have access to?
|
||||
# Assistant: I can see one file in the working directory: sales.csv.
|
||||
#
|
||||
# User: Read sales.csv and summarize what columns it contains and how many rows it has.
|
||||
# Assistant: sales.csv has 50 data rows and 7 columns: date, product, category,
|
||||
# quantity, unit_price, region, salesperson.
|
||||
#
|
||||
# User: Calculate the total revenue (quantity * unit_price) per region and show the result as a table.
|
||||
# Assistant:
|
||||
# | Region | Total Revenue |
|
||||
# |--------|---------------|
|
||||
# | North | $X,XXX.XX |
|
||||
# | South | $X,XXX.XX |
|
||||
# | West | $X,XXX.XX |
|
||||
#
|
||||
# User: Save a markdown report named region_totals.md ...
|
||||
# Assistant: I wrote region_totals.md to the working folder.
|
||||
#
|
||||
# User: List the files again so I can confirm region_totals.md was created.
|
||||
# Assistant: The working folder now contains: region_totals.md, sales.csv.
|
||||
#
|
||||
# Final contents of working/:
|
||||
# - region_totals.md (NNN bytes)
|
||||
# - sales.csv (3175 bytes)
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
date,product,category,quantity,unit_price,region,salesperson
|
||||
2025-01-03,Laptop Pro 15,Electronics,2,1299.99,North,Alice
|
||||
2025-01-05,Ergonomic Chair,Furniture,5,349.50,South,Bob
|
||||
2025-01-07,Wireless Mouse,Electronics,12,24.99,North,Alice
|
||||
2025-01-08,Standing Desk,Furniture,1,599.00,West,Carol
|
||||
2025-01-10,USB-C Hub,Electronics,8,45.99,North,David
|
||||
2025-01-12,Monitor 27in,Electronics,3,429.00,South,Bob
|
||||
2025-01-14,Desk Lamp,Furniture,6,79.95,West,Carol
|
||||
2025-01-15,Keyboard Mech,Electronics,4,149.99,North,Alice
|
||||
2025-01-17,Filing Cabinet,Furniture,2,189.00,South,David
|
||||
2025-01-20,Webcam HD,Electronics,10,89.99,West,Bob
|
||||
2025-01-22,Laptop Pro 15,Electronics,1,1299.99,South,Carol
|
||||
2025-01-24,Ergonomic Chair,Furniture,3,349.50,North,Alice
|
||||
2025-01-25,Notebook Pack,Stationery,20,12.99,South,David
|
||||
2025-01-27,Wireless Mouse,Electronics,15,24.99,West,Carol
|
||||
2025-01-28,Whiteboard,Stationery,4,129.00,North,Bob
|
||||
2025-01-30,Standing Desk,Furniture,2,599.00,South,Alice
|
||||
2025-02-02,USB-C Hub,Electronics,6,45.99,West,David
|
||||
2025-02-04,Monitor 27in,Electronics,2,429.00,North,Carol
|
||||
2025-02-05,Desk Lamp,Furniture,8,79.95,South,Bob
|
||||
2025-02-07,Keyboard Mech,Electronics,5,149.99,West,Alice
|
||||
2025-02-09,Filing Cabinet,Furniture,1,189.00,North,David
|
||||
2025-02-11,Webcam HD,Electronics,7,89.99,South,Carol
|
||||
2025-02-13,Laptop Pro 15,Electronics,3,1299.99,West,Bob
|
||||
2025-02-15,Notebook Pack,Stationery,30,12.99,North,Alice
|
||||
2025-02-17,Ergonomic Chair,Furniture,4,349.50,South,David
|
||||
2025-02-19,Wireless Mouse,Electronics,20,24.99,North,Carol
|
||||
2025-02-20,Whiteboard,Stationery,2,129.00,West,Bob
|
||||
2025-02-22,Standing Desk,Furniture,1,599.00,North,Alice
|
||||
2025-02-24,USB-C Hub,Electronics,10,45.99,South,David
|
||||
2025-02-26,Monitor 27in,Electronics,4,429.00,West,Carol
|
||||
2025-02-28,Desk Lamp,Furniture,3,79.95,North,Bob
|
||||
2025-03-02,Keyboard Mech,Electronics,6,149.99,South,Alice
|
||||
2025-03-04,Filing Cabinet,Furniture,3,189.00,West,David
|
||||
2025-03-06,Webcam HD,Electronics,9,89.99,North,Carol
|
||||
2025-03-08,Laptop Pro 15,Electronics,2,1299.99,South,Bob
|
||||
2025-03-10,Notebook Pack,Stationery,25,12.99,West,Alice
|
||||
2025-03-12,Ergonomic Chair,Furniture,6,349.50,North,David
|
||||
2025-03-14,Wireless Mouse,Electronics,18,24.99,South,Carol
|
||||
2025-03-15,Whiteboard,Stationery,5,129.00,North,Bob
|
||||
2025-03-17,Standing Desk,Furniture,3,599.00,West,Alice
|
||||
2025-03-19,USB-C Hub,Electronics,7,45.99,North,David
|
||||
2025-03-21,Monitor 27in,Electronics,5,429.00,South,Carol
|
||||
2025-03-23,Desk Lamp,Furniture,4,79.95,West,Bob
|
||||
2025-03-25,Keyboard Mech,Electronics,3,149.99,North,Alice
|
||||
2025-03-27,Filing Cabinet,Furniture,2,189.00,South,David
|
||||
2025-03-28,Webcam HD,Electronics,11,89.99,West,Carol
|
||||
2025-03-29,Laptop Pro 15,Electronics,1,1299.99,North,Bob
|
||||
2025-03-30,Notebook Pack,Stationery,15,12.99,South,Alice
|
||||
2025-03-31,Ergonomic Chair,Furniture,2,349.50,West,David
|
||||
|
@@ -8,6 +8,7 @@ filters the same way.
|
||||
"""
|
||||
|
||||
import concurrent.futures
|
||||
import contextlib
|
||||
import glob
|
||||
import os
|
||||
import subprocess
|
||||
@@ -17,6 +18,16 @@ from collections.abc import Sequence
|
||||
from fnmatch import fnmatch
|
||||
from pathlib import Path
|
||||
|
||||
# On Windows, stdout defaults to cp1252 under non-interactive callers (e.g.
|
||||
# prek / pre-commit hooks). Reconfigure to UTF-8 before importing rich so
|
||||
# unicode glyphs like ``\u2713`` don't raise ``UnicodeEncodeError``.
|
||||
if sys.platform == "win32":
|
||||
for _stream in (sys.stdout, sys.stderr):
|
||||
reconfigure = getattr(_stream, "reconfigure", None)
|
||||
if callable(reconfigure):
|
||||
with contextlib.suppress(OSError, ValueError):
|
||||
reconfigure(encoding="utf-8")
|
||||
|
||||
import tomli
|
||||
from rich import print
|
||||
|
||||
@@ -122,8 +133,7 @@ def project_filter_matches(project: Path | str, pattern: str, aliases: Sequence[
|
||||
"""
|
||||
normalized_pattern = normalize_project_filter(pattern).lower()
|
||||
return any(
|
||||
fnmatch(candidate, normalized_pattern)
|
||||
for candidate in build_project_filter_candidates(project, aliases)
|
||||
fnmatch(candidate, normalized_pattern) for candidate in build_project_filter_candidates(project, aliases)
|
||||
)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user