Python: replace pre-commit with prek, add PEP 723 script deps, clean up dev dependencies (#3748)

* python: replace pre-commit with prek, add PEP 723 script deps, clean up dev dependencies

- Replace pre-commit with prek (Rust-native, faster pre-commit alternative)
- Move supported hooks to repo: builtin for zero-clone speed
- Add new builtin hooks: trailing-whitespace, check-merge-conflict, detect-private-key, check-added-large-files
- Update all hook versions to latest (pre-commit-hooks v6, pyupgrade v3.21.2, bandit 1.9.3, uv-pre-commit 0.10.0)
- Add PEP 723 inline script metadata to 34 samples with external deps
- Remove autogen-agentchat/autogen-ext from dev deps (now declared per-sample)
- Remove unused dev deps: pytest-env, tomli-w
- Add agent-framework-core>=1.0.0b260130 lower bound to all 21 packages
- Update CI workflow to use j178/prek-action
- Update docs: DEV_SETUP.md, AGENTS.md, CODING_STANDARD.md, SAMPLE_GUIDELINES.md

* updated lock

* python: fix prek config paths for local execution and CI workflow

Remove global 'files: ^python/' filter and strip python/ prefix from all path patterns in .pre-commit-config.yaml so prek finds files when run from the python/ directory. Update CI workflow to use --cd python instead of --config path. Include trailing whitespace fixes and dev dependency cleanup.

* python: move helper scripts to scripts/ folder and exclude from checks

* python: exclude AGENTS.md from prek markdown code lint

* python: exclude AGENTS.md and azure_ai_search sample from markdown lint

* fix m365 sample

* python: ignore CPY rule for samples with PEP 723 headers

* fix in dev_setup

* python: replace aiofiles with regular open in samples

* python: suppress reportUnusedImport in markdown code block checker

* python: use samples pyright config for markdown code block checker

Write a temp pyrightconfig.json matching pyrightconfig.samples.json rules (typeCheckingMode=off, only reportMissingImports and reportAttributeAccessIssue). Filter output to only fail on these rules since syntax-level errors (top-level await, undefined vars) are expected in README documentation snippets.

* python: use markdown-code-lint with fixed globs instead of prek file list

The prek-markdown-code-lint task received all changed files including non-README markdown and files with pre-existing broken imports. Replace with the standard markdown-code-lint task which uses the correct glob patterns (README.md, packages/**/README.md, samples/**/*.md).

* python: exclude READMEs with pre-existing broken imports from markdown lint

* python: fix broken README code snippets instead of excluding them

- ag-ui: replace TextContent (removed) with content.type == 'text'
- durabletask: fix import path to durabletask.worker.TaskHubGrpcWorker
- orchestrations: use constructor params instead of .participants() method
- observability: mark deprecated code blocks as plain text, filter
  reportMissingImports to agent_framework modules only
- remove README excludes from markdown-code-lint task

* add revision to gaia download

* feat(python): parallelize checks across packages

Run (package × task) cross-product in parallel using ThreadPoolExecutor
and subprocesses. Key changes:

- Add scripts/task_runner.py with shared parallel execution engine
- Update run_tasks_in_packages_if_exists.py to accept multiple tasks
- Update run_tasks_in_changed_packages.py with --files flag and parallel support
- Add check-packages poe task (fmt+lint+pyright+mypy in parallel)
- Add prek-markdown-code-lint and prek-samples-check with change detection
- Split CI code quality workflow into parallel prek and mypy jobs
- Update DEV_SETUP.md to document new parallel behavior

Core package changes still trigger checks on all packages.

* feat(ci): split code quality into 4 parallel jobs

Split the single prek job into parallel jobs:
- pre-commit-hooks: lightweight hooks (SKIP=poe-check)
- package-checks: fmt/lint/pyright/mypy via check-packages
- samples-markdown: samples-lint, samples-syntax, markdown-code-lint
- mypy: change-detected mypy checks

All 4 jobs run concurrently (×2 Python versions = 8 runners).

* feat(ci): use only Python 3.10 for code quality checks

* refactor(python): add future annotations and remove quoted types

Add `from __future__ import annotations` to 93 package files that
used quoted string annotations, then run pyupgrade --py310-plus to
remove the now-unnecessary quotes.

Fixes https://github.com/microsoft/agent-framework/issues/3578

python/pyproject.toml

@@ -30,12 +30,10 @@ dependencies = [
 dev = [
     "uv>=0.9,<1.0.0",
     "flit>=3.12.0",
-    "pre-commit >= 3.7",
     "ruff>=0.11.8",
     "pytest>=8.4.1",
     "pytest-asyncio>=1.0.0",
     "pytest-cov>=6.2.1",
-    "pytest-env>=1.1.5",
     "pytest-xdist[psutil]>=3.8.0",
     "pytest-timeout>=2.3.1",
     "pytest-retry>=1",
@@ -45,16 +43,7 @@ dev = [
     "poethepoet>=0.36.0",
     "rich",
     "tomli",
-    "tomli-w",
-    # AutoGen migration samples
-    "autogen-agentchat",
-    "autogen-ext[openai]",
-]
-docs = [
-    # Documentation
-    "debugpy>=1.8.16",
-    "py2docfx>=0.1.22.dev2259826",
-    "pip",
+    "prek>=0.3.2",
 ]
 
 [tool.uv]
@@ -111,7 +100,7 @@ line-length = 120
 target-version = "py310"
 fix = true
 include = ["*.py", "*.pyi", "**/pyproject.toml", "*.ipynb"]
-exclude = ["docs/*", "run_tasks_in_packages_if_exists.py", "check_md_code_blocks.py"]
+exclude = ["scripts"]
 extend-exclude = [
     "[{][{]cookiecutter.package_name[}][}]",
 ]
@@ -157,7 +146,7 @@ ignore = [
 [tool.ruff.lint.per-file-ignores]
 # Ignore all directories named `tests` and `samples`.
 "**/tests/**" = ["D", "INP", "TD", "ERA001", "RUF", "S"]
-"samples/**" = ["D", "INP", "ERA001", "RUF", "S", "T201"]
+"samples/**" = ["D", "INP", "ERA001", "RUF", "S", "T201", "CPY"]
 "*.ipynb" = ["CPY", "E501"]
 
 [tool.ruff.format]
@@ -191,7 +180,7 @@ omit = [
 
 [tool.pyright]
 include = ["agent_framework*"]
-exclude = ["**/tests/**", "docs", "**/.venv/**", "packages/devui/frontend/**"]
+exclude = ["**/tests/**", "**/.venv/**", "packages/devui/frontend/**"]
 typeCheckingMode = "strict"
 reportUnnecessaryIsInstance = false
 reportMissingTypeStubs = false
@@ -212,35 +201,36 @@ disallow_untyped_decorators = true
 
 [tool.bandit]
 targets = ["agent_framework"]
-exclude_dirs = ["tests", "./run_tasks_in_packages_if_exists.py", "./check_md_code_blocks.py", "docs", "samples"]
+exclude_dirs = ["tests", "scripts", "samples"]
 
 [tool.poe]
 executor.type = "uv"
 
 [tool.poe.tasks]
-markdown-code-lint = "uv run python check_md_code_blocks.py 'README.md' './packages/**/README.md' './samples/**/*.md' --exclude cookiecutter-agent-framework-lab --exclude tau2 --exclude 'packages/devui/frontend'"
-pre-commit-install = "uv run pre-commit install --install-hooks --overwrite"
-install = "uv sync --all-packages --all-extras --dev -U --prerelease=if-necessary-or-explicit --no-group=docs"
-test = "python run_tasks_in_packages_if_exists.py test"
-fmt = "python run_tasks_in_packages_if_exists.py fmt"
+markdown-code-lint = "uv run python scripts/check_md_code_blocks.py 'README.md' './packages/**/README.md' './samples/**/*.md' --exclude cookiecutter-agent-framework-lab --exclude tau2 --exclude 'packages/devui/frontend' --exclude context_providers/azure_ai_search"
+prek-install = "prek install --overwrite"
+install = "uv sync --all-packages --all-extras --dev -U --prerelease=if-necessary-or-explicit"
+test = "python scripts/run_tasks_in_packages_if_exists.py test"
+fmt = "python scripts/run_tasks_in_packages_if_exists.py fmt"
 format.ref = "fmt"
-lint = "python run_tasks_in_packages_if_exists.py lint"
+lint = "python scripts/run_tasks_in_packages_if_exists.py lint"
 samples-lint = "ruff check samples --fix --exclude samples/autogen-migration,samples/semantic-kernel-migration --ignore E501,ASYNC,B901,TD002"
-pyright = "python run_tasks_in_packages_if_exists.py pyright"
-mypy = "python run_tasks_in_packages_if_exists.py mypy"
+pyright = "python scripts/run_tasks_in_packages_if_exists.py pyright"
+mypy = "python scripts/run_tasks_in_packages_if_exists.py mypy"
 samples-syntax = "pyright -p pyrightconfig.samples.json --warnings"
 typing = ["pyright", "mypy"]
 # cleaning
-clean-dist-packages = "python run_tasks_in_packages_if_exists.py clean-dist"
+clean-dist-packages = "python scripts/run_tasks_in_packages_if_exists.py clean-dist"
 clean-dist-meta = "rm -rf dist"
 clean-dist = ["clean-dist-packages", "clean-dist-meta"]
 # build and publish
-build-packages = "python run_tasks_in_packages_if_exists.py build"
+build-packages = "python scripts/run_tasks_in_packages_if_exists.py build"
 build-meta = "python -m flit build"
 build = ["build-packages", "build-meta"]
 publish = "uv publish"
 # combined checks
-check = ["fmt", "lint", "pyright", "mypy", "samples-lint", "samples-syntax", "test", "markdown-code-lint"]
+check-packages = "python scripts/run_tasks_in_packages_if_exists.py fmt lint pyright mypy"
+check = ["check-packages", "samples-lint", "samples-syntax", "test", "markdown-code-lint"]
 
 [tool.poe.tasks.all-tests-cov]
 cmd = """
@@ -285,16 +275,41 @@ args = [{ name = "python", default = "3.13", options = ['-p', '--python'] }]
 sequence = [
     { ref = "venv --python $python"},
     { ref = "install" },
-    { ref = "pre-commit-install" }
+    { ref = "prek-install" }
 ]
 args = [{ name = "python", default = "3.13", options = ['-p', '--python'] }]
 
-[tool.poe.tasks.pre-commit-markdown-code-lint]
-cmd = "uv run python check_md_code_blocks.py ${files} --no-glob --exclude cookiecutter-agent-framework-lab --exclude tau2 --exclude 'packages/devui/frontend'"
+[tool.poe.tasks.prek-pyright]
+cmd = "uv run python scripts/run_tasks_in_changed_packages.py pyright --files ${files}"
 args = [{ name = "files", default = ".", positional = true, multiple = true }]
 
-[tool.poe.tasks.pre-commit-pyright]
-cmd = "uv run python run_tasks_in_changed_packages.py pyright ${files}"
+[tool.poe.tasks.prek-check-packages]
+cmd = "uv run python scripts/run_tasks_in_changed_packages.py fmt lint pyright --files ${files}"
+args = [{ name = "files", default = ".", positional = true, multiple = true }]
+
+[tool.poe.tasks.prek-markdown-code-lint]
+cmd = """uv run python scripts/check_md_code_blocks.py ${files} --no-glob
+    --exclude cookiecutter-agent-framework-lab --exclude tau2
+    --exclude packages/devui/frontend --exclude context_providers/azure_ai_search"""
+args = [{ name = "files", default = ".", positional = true, multiple = true }]
+
+[tool.poe.tasks.prek-samples-check]
+shell = """
+HAS_SAMPLES=false
+for f in ${files}; do
+    case "$f" in
+        samples/*) HAS_SAMPLES=true; break ;;
+    esac
+done
+if [ "$HAS_SAMPLES" = true ]; then
+    echo "Sample files changed, running samples checks..."
+    uv run ruff check samples --fix --exclude samples/autogen-migration,samples/semantic-kernel-migration --ignore E501,ASYNC,B901,TD002
+    uv run pyright -p pyrightconfig.samples.json --warnings
+else
+    echo "No sample files changed, skipping samples checks"
+fi
+"""
+interpreter = "bash"
 args = [{ name = "files", default = ".", positional = true, multiple = true }]
 
 
@@ -316,18 +331,15 @@ else
                     echo ".")
 fi
 echo "Changed files: $CHANGED_FILES"
-uv run python run_tasks_in_changed_packages.py mypy $CHANGED_FILES
+uv run python scripts/run_tasks_in_changed_packages.py mypy --files $CHANGED_FILES
 """
 interpreter = "bash"
 
-[tool.poe.tasks.pre-commit-check]
+[tool.poe.tasks.prek-check]
 sequence = [
-    { ref = "fmt" },
-    { ref = "lint" },
-    { ref = "pre-commit-pyright ${files}" },
-    { ref = "pre-commit-markdown-code-lint ${files}" },
-    { ref = "samples-lint" },
-    { ref = "samples-syntax" }
+    { ref = "prek-check-packages ${files}" },
+    { ref = "prek-markdown-code-lint ${files}" },
+    { ref = "prek-samples-check ${files}" }
 ]
 args = [{ name = "files", default = ".", positional = true, multiple = true }]