jif-oai 917a9a41a3 Parallelize cold rollout compression (#25654) ...

## Why

[#25089](https://github.com/openai/codex/pull/25089) added the
background worker for compressing cold archived rollouts, but the worker
still processed files effectively one at a time: each compression job
was sent to `spawn_blocking` and then awaited before the next file
started. On machines with a backlog of archived rollouts, that makes
catch-up slower than it needs to be even though the actual compression
work already runs off the async runtime.

## What Changed

- Queue rollout compression work in a `JoinSet` while directory
traversal continues.
- Cap the worker at two in-flight compression jobs so it can overlap
compression without turning the background task into unbounded blocking
work.
- Drain pending jobs before returning, including the
`read_dir.next_entry()` error path, so every launched job still
contributes to the final `compressed`, `skipped`, and `failed` stats.
- Treat task join failures the same way as compression failures in the
worker's warning and failure accounting.

917a9a41a3 · 2026-06-01 19:54:52 +02:00

History

..

src

Parallelize cold rollout compression (#25654)

2026-06-01 19:54:52 +02:00

BUILD.bazel

Extract rollout into its own crate (#15548)

2026-03-24 18:10:53 -07:00

Cargo.toml

Read compressed rollouts and materialize before append (#25087)

2026-06-01 15:14:19 +02:00