3 Commits

• Python: Adding AgentFileStore and FileAccessProvider to support file access operations. (#6099)

  * Adding AgentFileStore and FileAccessProvider to support file ased operations for agents.
  
  * Address PR review feedback on FileAccessProvider
  
  - Probe symlinks on the unresolved candidate path so in-root symlinks
    cannot silently pass and out-of-root symlinks surface the correct
    error message.
  - Validate matching_lines elements in FileSearchResult.from_dict and
    raise a clean ValueError for non-mapping entries.
  - Cap search regex pattern length (256 chars) via a new
    _compile_search_regex helper to mitigate ReDoS, and surface the cap
    in the file_access_search_files tool description.
  - Skip non-UTF-8 files during filesystem search instead of aborting
    the entire directory walk.
  - Replace the module-scope trailing string in the data-processing
    sample with comments to avoid Ruff B018.
  - Remove the checked-in working/region_totals.md sample artifact so
    the save flow works from a clean checkout.
  - Expand the Windows stdout reconfiguration comment in task_runner.py
    for clarity.
  - Add tests for invalid/oversize regex, non-UTF-8 file search, and
    in-root symlink rejection.
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
  
  * Fix mypy redundant-cast in FileSearchResult.from_dict
  
  Use cast(list[object], ...) instead of cast(list[Any], ...) so the
  cast represents a real type change (lists are invariant) and is no
  longer flagged by mypy as redundant, while still satisfying pyright's
  reportUnknownVariableType. Matches the existing pattern in _memory.py.
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
  
  * Tighten path normalization and directory resolution in FileAccess
  
  - _normalize_relative_path now strips surrounding whitespace up front
    so leading/trailing spaces never leak into file segments, and
    rejects trailing path separators for file paths so 'foo/' is no
    longer silently coerced to 'foo'.
  - FileSystemAgentFileStore._resolve_safe_directory_path normalizes
    with is_directory=True and maps an empty normalized result to the
    root. This matches InMemoryAgentFileStore so whitespace-only
    directory inputs resolve to the root instead of raising.
  - Added tests for whitespace stripping, trailing-separator rejection,
    and whitespace-only directory listing on the filesystem store.
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
  
  * Harden FileAccess search and atomic save in store API
  
  - Add wall-clock timeout (10s) around regex scans so a pathological pattern (e.g. `(a+)+`) below the length cap cannot stall the event loop.
  - Offload the InMemoryAgentFileStore regex scan to a worker thread, matching the filesystem store.
  - Fail closed when `Path.is_symlink` raises during the safe-path probe so a permission error cannot silently bypass the symlink/reparse-point rejection.
  - Add `overwrite: bool = True` to `AgentFileStore.write_file`; the in-memory store performs the check under the existing lock and the filesystem store uses `open(mode='x')` so concurrent callers cannot race past `overwrite=False`.
  - `file_access_save_file` now relies on the atomic store call instead of a separate `file_exists` round-trip.
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
  
  * Fix Python 3.10 timeout handling and add directory arg to list/search tools
  
  - Catch asyncio.TimeoutError in _run_search_with_timeout. In Python 3.10
    asyncio.wait_for raises asyncio.exceptions.TimeoutError, which is
    distinct from the builtin TimeoutError (the two were unified in 3.11).
    Catching the asyncio alias works on every supported version.
  - Add an optional directory parameter to file_access_list_files and
    file_access_search_files so agents can enumerate / scope searches to
    nested folders, not just the store root.
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
  
  * Address FileAccess review feedback: case, errors, signal, TOCTOU
  
  - InMemoryAgentFileStore now stores (display_name, content) so list_files
    and search_files return the original-case names callers wrote, matching
    the behaviour of FileSystemAgentFileStore on case-preserving filesystems
    and removing the silent in-memory vs. on-disk contract divergence.
  - FileSystemAgentFileStore.read_file raises ValueError instead of letting
    UnicodeDecodeError bubble for binary / non-UTF-8 input, restoring
    symmetry with search_files (which still skips) and giving the tool
    layer a recoverable type to translate.
  - Tool wrappers now catch ValueError and OSError around every operation
    and surface them as readable strings, so 'you used ..' and 'the file
    already exists' are both reported to the model the same way instead of
    the former crashing out as an unhandled exception.
  - _search_files_sync logs per skipped non-UTF-8 file at WARNING and an
    aggregate INFO summary so operators can distinguish 'no matches' from
    'half the corpus was unreadable'.
  - FileSystemAgentFileStore softens its docstrings to acknowledge the
    inherent probe-then-open TOCTOU window. On POSIX both read and write
    now pass O_NOFOLLOW so the kernel refuses if the leaf segment becomes
    a symlink between the probe and the open. Windows has no equivalent
    flag; the limitation is documented.
  - Tests cover: case preservation on list/search, ValueError on non-UTF-8
    read at the store and tool layer, tool-layer string responses for
    path-traversal and oversized-regex inputs, search-skip log output,
    symlink rejection on delete/search/list, and symlinked intermediate
    directory rejection.
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
  
  * Address FileAccess nit comments: docstrings, enumerate, opt-in delete approval
  
  - Expand FileSearchMatch/FileSearchResult.to_dict docstrings to explain why
    the override is needed (__slots__ defeats the mixin's __dict__ iteration)
    and why exclude/exclude_none are accepted-but-ignored (mixin signature
    compatibility for callers like to_json).
  - Use enumerate(lines, start=1) in _search_file_content so the +1 below is
    no longer needed; rename loop variable to line_number for clarity.
  - Add opt-in require_delete_approval: bool = False on FileAccessProvider.
    When True, file_access_delete_file is registered with approval_mode
    'always_require' so the host must approve every delete. Default False
    preserves current behaviour and matches the .NET reference, but
    deployments that want a safer-by-default posture can enable it.
  - Add tests covering both delete approval modes.
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
  
  * FileAccess: require delete approval by default
  
  Flip the default for FileAccessProvider(require_delete_approval=...) from
  False to True so destructive deletes are gated by host approval out of the
  box. Callers that want the previous autonomous behaviour (which matches the
  .NET reference) can pass require_delete_approval=False.
  
  Tests updated accordingly.
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
  
  * Fixing linkinspector by installing Chrome for puppeteer first.
  
  ---------
  
  Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com>
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

  Ben Thomas · 2026-05-28 20:09:50 +00:00
  b000a2cf51
• Python: Simplify Python Poe tasks and unify package selectors (#4722)

  * updated automation tasks and commands, with alias for the time being
  
  * Restore aggregate test exclusions
  
  Preserve the legacy all-tests scope for test --all by excluding lab and devui from the default aggregate sweep, while still allowing explicit package selection. Also ignore hidden/generated test directories such as .mypy_cache during aggregate discovery.
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
  
  * updated versions in pre-commit
  
  ---------
  
  Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

  Eduard van Valkenburg · 2026-03-18 18:39:11 +00:00
  f48c4512d3
• Python: replace pre-commit with prek, add PEP 723 script deps, clean up dev dependencies (#3748)

  * python: replace pre-commit with prek, add PEP 723 script deps, clean up dev dependencies
  
  - Replace pre-commit with prek (Rust-native, faster pre-commit alternative)
  - Move supported hooks to repo: builtin for zero-clone speed
  - Add new builtin hooks: trailing-whitespace, check-merge-conflict, detect-private-key, check-added-large-files
  - Update all hook versions to latest (pre-commit-hooks v6, pyupgrade v3.21.2, bandit 1.9.3, uv-pre-commit 0.10.0)
  - Add PEP 723 inline script metadata to 34 samples with external deps
  - Remove autogen-agentchat/autogen-ext from dev deps (now declared per-sample)
  - Remove unused dev deps: pytest-env, tomli-w
  - Add agent-framework-core>=1.0.0b260130 lower bound to all 21 packages
  - Update CI workflow to use j178/prek-action
  - Update docs: DEV_SETUP.md, AGENTS.md, CODING_STANDARD.md, SAMPLE_GUIDELINES.md
  
  * updated lock
  
  * python: fix prek config paths for local execution and CI workflow
  
  Remove global 'files: ^python/' filter and strip python/ prefix from all path patterns in .pre-commit-config.yaml so prek finds files when run from the python/ directory. Update CI workflow to use --cd python instead of --config path. Include trailing whitespace fixes and dev dependency cleanup.
  
  * python: move helper scripts to scripts/ folder and exclude from checks
  
  * python: exclude AGENTS.md from prek markdown code lint
  
  * python: exclude AGENTS.md and azure_ai_search sample from markdown lint
  
  * fix m365 sample
  
  * python: ignore CPY rule for samples with PEP 723 headers
  
  * fix in dev_setup
  
  * python: replace aiofiles with regular open in samples
  
  * python: suppress reportUnusedImport in markdown code block checker
  
  * python: use samples pyright config for markdown code block checker
  
  Write a temp pyrightconfig.json matching pyrightconfig.samples.json rules (typeCheckingMode=off, only reportMissingImports and reportAttributeAccessIssue). Filter output to only fail on these rules since syntax-level errors (top-level await, undefined vars) are expected in README documentation snippets.
  
  * python: use markdown-code-lint with fixed globs instead of prek file list
  
  The prek-markdown-code-lint task received all changed files including non-README markdown and files with pre-existing broken imports. Replace with the standard markdown-code-lint task which uses the correct glob patterns (README.md, packages/**/README.md, samples/**/*.md).
  
  * python: exclude READMEs with pre-existing broken imports from markdown lint
  
  * python: fix broken README code snippets instead of excluding them
  
  - ag-ui: replace TextContent (removed) with content.type == 'text'
  - durabletask: fix import path to durabletask.worker.TaskHubGrpcWorker
  - orchestrations: use constructor params instead of .participants() method
  - observability: mark deprecated code blocks as plain text, filter
    reportMissingImports to agent_framework modules only
  - remove README excludes from markdown-code-lint task
  
  * add revision to gaia download
  
  * feat(python): parallelize checks across packages
  
  Run (package × task) cross-product in parallel using ThreadPoolExecutor
  and subprocesses. Key changes:
  
  - Add scripts/task_runner.py with shared parallel execution engine
  - Update run_tasks_in_packages_if_exists.py to accept multiple tasks
  - Update run_tasks_in_changed_packages.py with --files flag and parallel support
  - Add check-packages poe task (fmt+lint+pyright+mypy in parallel)
  - Add prek-markdown-code-lint and prek-samples-check with change detection
  - Split CI code quality workflow into parallel prek and mypy jobs
  - Update DEV_SETUP.md to document new parallel behavior
  
  Core package changes still trigger checks on all packages.
  
  * feat(ci): split code quality into 4 parallel jobs
  
  Split the single prek job into parallel jobs:
  - pre-commit-hooks: lightweight hooks (SKIP=poe-check)
  - package-checks: fmt/lint/pyright/mypy via check-packages
  - samples-markdown: samples-lint, samples-syntax, markdown-code-lint
  - mypy: change-detected mypy checks
  
  All 4 jobs run concurrently (×2 Python versions = 8 runners).
  
  * feat(ci): use only Python 3.10 for code quality checks
  
  * refactor(python): add future annotations and remove quoted types
  
  Add `from __future__ import annotations` to 93 package files that
  used quoted string annotations, then run pyupgrade --py310-plus to
  remove the now-unnecessary quotes.
  
  Fixes https://github.com/microsoft/agent-framework/issues/3578

  Eduard van Valkenburg · 2026-02-09 17:51:01 +00:00
  977c3adfb2