Python: Simplify Python Poe tasks and unify package selectors (#4722)

* updated automation tasks and commands, with alias for the time being

* Restore aggregate test exclusions

Preserve the legacy all-tests scope for test --all by excluding lab and devui from the default aggregate sweep, while still allowing explicit package selection. Also ignore hidden/generated test directories such as .mypy_cache during aggregate discovery.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updated versions in pre-commit

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Eduard van Valkenburg
2026-03-18 19:39:11 +01:00
committed by GitHub
Unverified
parent d3d0100822
commit f48c4512d3
60 changed files with 1704 additions and 527 deletions
+26 -16
View File
@@ -13,26 +13,34 @@ description: >
All commands run from the `python/` directory:
```bash
# Format code (ruff format, parallel across packages)
uv run poe fmt
# Lint and auto-fix (ruff check, parallel across packages)
uv run poe lint
# Syntax formatting + checks (parallel across packages by default)
uv run poe syntax
uv run poe syntax -P core
uv run poe syntax -F # Format only
uv run poe syntax -C # Check only
uv run poe syntax -S # Samples only
# Type checking
uv run poe pyright # Pyright (parallel across packages)
uv run poe mypy # MyPy (parallel across packages)
uv run poe pyright # Pyright fan-out across packages
uv run poe pyright -P core
uv run poe pyright -A
uv run poe mypy # MyPy fan-out across packages
uv run poe mypy -P core
uv run poe mypy -A
uv run poe typing # Both pyright and mypy
uv run poe typing -P core
uv run poe typing -A
# All package-level checks in parallel (fmt + lint + pyright + mypy)
# All package-level checks in parallel (syntax + pyright)
uv run poe check-packages
# Full check (packages + samples + tests + markdown)
uv run poe check
uv run poe check -P core
# Samples only
uv run poe samples-lint # Ruff lint on samples/
uv run poe samples-syntax # Pyright syntax check on samples/
uv run poe check -S
uv run poe pyright -S
# Markdown code blocks
uv run poe markdown-code-lint
@@ -40,8 +48,8 @@ uv run poe markdown-code-lint
## Pre-commit Hooks (prek)
Prek hooks run automatically on commit. They check only changed files and run
package-level checks in parallel for affected packages only.
Prek hooks run automatically on commit. They stay lightweight and only check
changed files.
```bash
# Install hooks
@@ -54,8 +62,10 @@ uv run prek run -a
uv run prek run --last-commit
```
When core package changes, type-checking (mypy, pyright) runs across all packages
since type changes propagate. Format and lint only run in changed packages.
They run changed-package syntax formatting/checking, markdown code lint only
when markdown files change, and sample syntax lint/pyright only when files
under `samples/` change.
They intentionally do not run workspace `pyright` or `mypy` by default.
## Ruff Configuration
@@ -80,6 +90,6 @@ in-process with streaming output.
CI splits into 4 parallel jobs:
1. **Pre-commit hooks** — lightweight hooks (SKIP=poe-check)
2. **Package checks**fmt/lint/pyright via check-packages
3. **Samples & markdown**samples-lint, samples-syntax, markdown-code-lint
2. **Package checks**syntax/pyright via check-packages
3. **Samples & markdown**`check -S` plus `markdown-code-lint`
4. **Mypy** — change-detected mypy checks
+9 -9
View File
@@ -47,17 +47,17 @@ uv run poe upgrade-dev-dependencies
# First, run workspace-wide lower/upper compatibility gates
uv run poe validate-dependency-bounds-test
# Defaults to --project "*"; pass a package to scope test mode
uv run poe validate-dependency-bounds-test --project <workspace-package-name>
# Defaults to --package "*"; pass a package to scope test mode
uv run poe validate-dependency-bounds-test --package core
# Then expand bounds for one dependency in the target package
uv run poe validate-dependency-bounds-project --mode both --project <workspace-package-name> --dependency "<dependency-name>"
uv run poe validate-dependency-bounds-project --mode both --package core --dependency "<dependency-name>"
# Repo-wide automation can reuse the same task
uv run poe validate-dependency-bounds-project --mode upper --project "*"
uv run poe validate-dependency-bounds-project --mode upper --package "*"
# Add a dependency to one project and run both validators for that project/dependency
uv run poe add-dependency-and-validate-bounds --project <workspace-package-name> --dependency "<dependency-spec>"
uv run poe add-dependency-and-validate-bounds --package core --dependency "<dependency-spec>"
```
### Dependency Bound Notes
@@ -66,7 +66,7 @@ uv run poe add-dependency-and-validate-bounds --project <workspace-package-name>
- Prerelease (`dev`/`a`/`b`/`rc`) and `<1.0` dependencies should use hard bounds with an explicit upper cap (avoid open-ended ranges).
- For `<1.0` dependencies, prefer the broadest validated range the package can really support. That may be a patch line, a minor line, or multiple minor lines when checks/tests show the broader lane is compatible.
- Prefer supporting multiple majors when practical; if APIs diverge across supported majors, use version-conditional imports/paths.
- For dependency changes, run workspace-wide bound gates first, then `validate-dependency-bounds-project --mode both` for the target package/dependency to keep minimum and maximum constraints current. The same task can also drive repo-wide upper-bound automation by using `--project "*"` and omitting `--dependency`.
- For dependency changes, run workspace-wide bound gates first, then `validate-dependency-bounds-project --mode both` for the target package/dependency to keep minimum and maximum constraints current. The same task can also drive repo-wide upper-bound automation by using `--package "*"` and omitting `--dependency`.
- Prefer targeted lock updates with `uv lock --upgrade-package <dependency-name>` to reduce `uv.lock` merge conflicts.
- Use `add-dependency-and-validate-bounds` for package-scoped dependency additions plus bound validation in one command.
- Use `upgrade-dev-dependencies` for repo-wide dev tooling refreshes; it repins dev dependencies, refreshes `uv.lock`, and reruns `check`, `typing`, and `test`.
@@ -108,12 +108,12 @@ def __getattr__(name: str) -> Any:
Recommended dependency workflow during connector implementation:
1. Add the dependency to the target package:
`uv run poe add-dependency-to-project --project <workspace-package-name> --dependency "<dependency-spec>"`
`uv run poe add-dependency-to-project --package core --dependency "<dependency-spec>"`
2. Implement connector code and tests.
3. Validate dependency bounds for that package/dependency:
`uv run poe validate-dependency-bounds-project --mode both --project <workspace-package-name> --dependency "<dependency-name>"`
`uv run poe validate-dependency-bounds-project --mode both --package core --dependency "<dependency-name>"`
4. If the package has meaningful tests/checks that validate dependency compatibility, you can use the add + validation flow in one command:
`uv run poe add-dependency-and-validate-bounds --project <workspace-package-name> --dependency "<dependency-spec>"`
`uv run poe add-dependency-and-validate-bounds --package core --dependency "<dependency-spec>"`
If compatibility checks are not in place yet, add the dependency first, then implement tests before running bound validation.
### Promotion to Stable
+7 -4
View File
@@ -41,11 +41,14 @@ Do **not** add sample-only dependencies to the root `pyproject.toml` dev group.
## Syntax Checking
```bash
# Check samples for syntax errors and missing imports
uv run poe samples-syntax
# Format + lint samples
uv run poe syntax -S
# Lint samples
uv run poe samples-lint
# Check samples for syntax errors and missing imports
uv run poe pyright -S
# Lint samples only
uv run poe syntax -S -C
```
## Documentation
+15 -8
View File
@@ -17,20 +17,27 @@ We run tests in two stages, for a PR each commit is tested with unit tests only
# Run tests for all packages in parallel
uv run poe test
# Run tests for a specific package
uv run --directory packages/core poe test
# Run tests for a specific workspace package
uv run poe test -P core
# Run all tests in a single pytest invocation (faster, uses pytest-xdist)
uv run poe all-tests
# Run all selected tests in a single pytest invocation
uv run poe test -A
# With coverage
uv run poe all-tests-cov
uv run poe test -A -C
uv run poe test -P core -C
# Run only unit tests (exclude integration tests)
uv run poe all-tests -m "not integration"
uv run poe test -A -m "not integration"
# Run only integration tests
uv run poe all-tests -m integration
uv run poe test -A -m integration
```
Direct package execution still works when you need it:
```bash
uv run --directory packages/core poe test
```
## Test Configuration
@@ -38,7 +45,7 @@ uv run poe all-tests -m integration
- **Async mode**: `asyncio_mode = "auto"` is enabled — do NOT use `@pytest.mark.asyncio`, but do mark tests with `async def` and use `await` for async calls
- **Timeout**: Default 60 seconds per test
- **Import mode**: `importlib` for cross-package isolation
- **Parallelization**: Large packages (core, ag-ui, orchestrations, anthropic) use `pytest-xdist` (`-n auto --dist worksteal`) in their `poe test` task. The `all-tests` task also uses xdist across all packages.
- **Parallelization**: Large packages (core, ag-ui, orchestrations, anthropic) use `pytest-xdist` (`-n auto --dist worksteal`) in their `poe test` task. The aggregate `uv run poe test -A` sweep also uses xdist across the selected packages.
## Test Directory Structure