mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
[codex] Speed up local nextest runs (#26479)
## Why `just test` currently uses the CI-oriented nextest profile, which serializes app-server integration tests even on developer machines that can run several safely. Bounded local parallelism substantially shortens this common iteration loop without changing CI behavior. Eight-worker experiments were faster, but keeping them reliable required relaxing several test deadlines. Four workers for integration tests is a solid tradeoff that speeds up local testing without needing to change test logic. ## What changed - Add a `local` nextest profile that inherits the existing defaults. - Allow up to four app-server integration tests to run concurrently under that profile. - Make `just test` select the local profile on Unix and Windows. - Keep the default CI profile serialized and leave all test deadlines unchanged. The tests use separate processes, randomized temporary `CODEX_HOME` directories, and ephemeral ports. The remaining shared constraints are system resources; each app-server also uses a multi-thread Tokio runtime, and fuzzy-search tests can create additional worker threads, so the local cap remains intentionally conservative. ## Performance and validation All measurements below are warm, execution-only app-server runs with nextest retries disabled. On the current rebased branch, an AMD EPYC 7763 machine with 16 logical CPUs and 62 GiB RAM completed three consecutive runs: | Run | Nextest time | Wall time | Result | | --- | ---: | ---: | --- | | 1 | 142.941s | 145.17s | 836/836 passed | | 2 | 143.402s | 145.59s | 836/836 passed | | 3 | 142.870s | 145.08s | 836/836 passed | The mean wall time was 145.28s. The slow-inventory, approval replay, and zsh-fork tests all passed with their original deadlines. Earlier measurements on the same Linux machine, before the suite grew, showed the scaling that motivated the change: | App-server concurrency | Nextest time | Result | | --- | ---: | --- | | 1 | 369.5s | 572/572 passed | | 2 | 194.5s | 572/572 passed | | 4 | 111.0s mean over 3 runs | 3/3 clean | Four workers reduced that execution time by about 70%, a roughly 3.3x speedup over serialization.
This commit is contained in:
committed by
GitHub
Unverified
parent
a71e040df5
commit
6e7ab52929
@@ -8,12 +8,20 @@ retries = 1
|
||||
[profile.default.junit]
|
||||
path = "junit.xml"
|
||||
|
||||
[profile.local]
|
||||
inherits = "default"
|
||||
|
||||
[test-groups.app_server_protocol_codegen]
|
||||
max-threads = 1
|
||||
|
||||
[test-groups.app_server_integration]
|
||||
max-threads = 1
|
||||
|
||||
# Higher concurrency causes integration test timeouts under resource contention
|
||||
# on common developer machines.
|
||||
[test-groups.app_server_integration_local]
|
||||
max-threads = 4
|
||||
|
||||
[test-groups.core_apply_patch_cli_integration]
|
||||
max-threads = 1
|
||||
|
||||
@@ -42,6 +50,12 @@ test-group = 'app_server_protocol_codegen'
|
||||
filter = 'package(codex-app-server) & kind(test)'
|
||||
test-group = 'app_server_integration'
|
||||
|
||||
[[profile.local.overrides]]
|
||||
# Use up to four app-server subprocesses locally. The global nextest pool still
|
||||
# limits this to the machine's logical CPU count.
|
||||
filter = 'package(codex-app-server) & kind(test)'
|
||||
test-group = 'app_server_integration_local'
|
||||
|
||||
[[profile.default.overrides]]
|
||||
# These tests exercise full Codex turns and apply_patch execution, and they are
|
||||
# sensitive to Windows runner process-startup stalls when many cases launch at once.
|
||||
|
||||
@@ -75,11 +75,11 @@ install:
|
||||
# there should be no need to add `--all-features`.
|
||||
[unix]
|
||||
test *args:
|
||||
RUST_MIN_STACK={{ rust_min_stack }} cargo nextest run --no-fail-fast "$@"
|
||||
RUST_MIN_STACK={{ rust_min_stack }} NEXTEST_PROFILE=local cargo nextest run --no-fail-fast "$@"
|
||||
|
||||
[windows]
|
||||
test *args:
|
||||
$env:RUST_MIN_STACK = "{{ rust_min_stack }}"; cargo nextest run --no-fail-fast @($args | Select-Object -Skip 1)
|
||||
$env:RUST_MIN_STACK = "{{ rust_min_stack }}"; $env:NEXTEST_PROFILE = "local"; cargo nextest run --no-fail-fast @($args | Select-Object -Skip 1)
|
||||
|
||||
# Run from the repository root so scripts that resolve paths from `cwd` see
|
||||
# the same layout they use in GitHub Actions.
|
||||
|
||||
Reference in New Issue
Block a user