mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
a0d5fd772e5dbc1be57c5a689cc4db1eabe5035d
22 Commits
-
core: add wait_for_environment for starting environments (#29745)
## Why With `DeferredExecutor`, a sampling request can begin while an environment is still starting. The model can see that pending state, but needs a way to wait for the environment within the same turn before continuing. Environment startup is owned by Core, so the wait tool should use the same request-frozen `StepContext` that advertised the starting environment. This keeps tool registration and execution tied to the exact startup operation the model saw, even if live thread state later changes. Supersedes #29735. ## What - register `wait_for_environment` when the current `StepContext` contains starting environments - wait on the selected `StartingTurnEnvironment` shared resolution and return a bounded ready or failed result - rebuild the next request normally, removing the wait tool and exposing ready environment tools, or reporting the environment as unavailable after failure ## Testing - `just test -p codex-core deferred_executor_` - verifies the wait tool is replaced by environment-backed tools after startup - verifies startup failure removes both the wait tool and unavailable environment tools while notifying the model
sayan-oai ·
2026-06-24 00:35:34 +00:00 -
[3/3] app-server: configure environment connection timeout (#29025)
## Why Remote environments registered through `environment/add` currently use the fixed 10-second WebSocket connection timeout. Slow-starting executors need a caller-selected connection window, but this should not add retry policy or couple exec-server behavior to Core’s `deferred_executor` feature. Make the timeout an optional part of the existing experimental request. Existing clients continue using the current default, while callers that know an executor may take longer can request a larger window explicitly. Depends on #28683. ## What changed - Add optional `connectTimeoutMs` to `EnvironmentAddParams` and document it in the app-server README. - Pass the optional timeout through `EnvironmentRequestProcessor` into one `EnvironmentManager::upsert_environment()` path; the manager applies the existing default when it is omitted. - Preserve the existing single-attempt lifecycle. The configured value controls WebSocket connection and handshake time for both initial connection and later reconnects; initialization retains its separate timeout. - Add an app-server integration test that sends the real JSON-RPC request and verifies a stalled handshake observes the requested timeout. ## Test plan - `just test -p codex-app-server-protocol` - `just test -p codex-exec-server` - `just test -p codex-app-server environment_add_applies_connect_timeout` ## Rollout This is additive and does not enable `deferred_executor`. Callers should send a non-default timeout only after a compatible app-server is deployed; omitted or `null` values retain the existing 10-second default.
sayan-oai ·
2026-06-19 05:27:45 +00:00 -
[2/3] core: track starting environments in snapshots (#28683)
## Why Remote environments may still be resolving when Codex creates a session or turn. Waiting for the existing all-or-nothing environment snapshot can hold startup until the selected environment is usable. Behind the default-off `deferred_executor` feature, let callers take a useful snapshot immediately: completed environments remain available normally, while unfinished environments are reported without blocking startup. With the feature disabled, snapshots preserve the existing blocking behavior. Depends on #28674. ## What changed - Store one ordered list of selected environments in `ThreadEnvironments`. Each selection owns one shared resolution that produces its complete `TurnEnvironment`. - Start new resolutions in the background with `remote_handle()`, allowing snapshots and the future wait tool to share the same result while cancellation follows the retained handles. - Make `snapshot()` a read-only operation: nonblocking snapshots collect completed resolutions and retain handles for unfinished ones, while blocking snapshots await every resolution. - Replace completed failed resolutions from the current manager entry and log when failed environments are omitted. - Return attached and starting environments as a point-in-time view, and count starting environments when deciding whether a snapshot is local-only. - Keep existing consumers attached-only. `to_selections()` derives from attached environments, so child threads do not inherit an environment that is still starting. ## Test plan - `just test -p codex-core environment_selection` - `just test -p codex-core deferred_executor_reaches_model_before_remote_environment_is_ready` ## Landing note Keep `deferred_executor` disabled for slow-starting executors until configurable `environment/add` connection timeouts and caller support land. When enabled, an environment that attaches after session startup may remain absent from environment-derived model context, tools, instructions, skills, and related state until follow-up refresh work lands.
sayan-oai ·
2026-06-19 05:06:34 +00:00 -
app-server: preserve target-native environment cwd (#28146)
## Why app-server may run on a different OS from the selected exec-server environment. Parsing that environment’s cwd with the Codex host’s path rules prevents thread startup. ## What Carry environment cwd values as `LegacyAppPathString` at the app-server boundary and `PathUri` internally. Existing tool-call schemas and relative-path behavior stay host-native; remaining local-only consumers convert explicitly and leave follow-up TODOs. The Wine integration test verifies app-server can start a thread and complete an ordinary turn with a Windows environment cwd from Linux. ## Validation - `bazel test //codex-rs/core/tests/remote_env_windows:smoke-test --test_output=errors` - focused app-server environment-selection and protocol schema tests - scoped Clippy for `codex-core` and `codex-app-server-protocol`
Adam Perry @ OpenAI ·
2026-06-16 21:42:28 +00:00 -
[codex] Use local environment for user shell commands (#28163)
## Why User shell commands still read the legacy turn cwd and session shell even though execution context is now owned by selected turn environments. App-server also defines `thread/shellCommand` as a local-host escape hatch, so it must use an available local environment even when a remote environment is primary. ## What changed - Add `ResolvedTurnEnvironments::local()` to find the selected local environment. - Resolve the user shell command cwd and shell from that local `TurnEnvironment`. - Emit the standard `shell is unavailable in this session` error when no selected local environment or resolved local shell is available. - Add an integration test covering `/shell` without a local environment. ## Test plan - `just test -p codex-core user_shell_command_without_local_environment_emits_error`
pakrym-oai ·
2026-06-16 04:55:20 +00:00 -
[codex] Bind shell snapshots to retained thread environments (#28421)
## Why Shell snapshots are currently session-scoped even though shell and cwd are properties of a selected turn environment. That makes snapshot refresh depend on separate session-cwd plumbing, prevents retained environments from retaining their snapshot work, and can make snapshot construction use a different shell than command execution. This follows #27955 by making the retained thread-environment service own environment snapshot lifecycles. Session configuration remains the requested selection state, while `ThreadEnvironments` remains the source of successfully resolved environments. ## What changed - Configure the shell-snapshot builder before initial environment resolution. - Start each local environment snapshot task when its `TurnEnvironment` is built and retain that shared task while environment ID and cwd still match. - Inherit retained environment snapshots into spawned child threads. - Carry the selected `TurnEnvironment` through shell runtimes so snapshot construction and command execution use the same environment-specific shell and cwd. - Load project instructions and warm plugins/skills after initial environment resolution. - Continue decoding invalid UTF-8 instruction files lossily without emitting a startup warning. - Keep requested selections in `SessionConfiguration`; failed or duplicate resolutions only affect the resolved environment snapshot. ## Validation - `cargo check -p codex-core --tests` - `just test -p codex-home instructions` (6 passed) - Focused environment, instruction, shell-snapshot, and user-shell tests (84 passed) - Focused shell-snapshot, user-shell, and unified-exec tests (126 passed; two event-timing tests passed on retry)
pakrym-oai ·
2026-06-15 20:10:53 -07:00 -
[codex] retain resolved environments across turns (#27955)
## Why Selected execution environments are thread-scoped resources, but startup and turn construction repeatedly resolved their IDs and working directories. That discarded existing environment handles and shell metadata even when a selection had not changed. Session configuration updates also need to affect future turns without changing the resolved environment set already captured by a running turn. ## What changed - Create a `ThreadEnvironments` service inside `Codex` from the spawned `EnvironmentManager` and raw environment selections, then store it on `SessionServices`. - Split service construction from `update_selections`, allowing session configuration updates to mutate the resolved set in place. - Retain an existing `TurnEnvironment` when its environment ID and working directory match; resolve only added or changed selections and remove selections that are no longer present. - Normalize duplicate IDs by keeping the first selection and skip individual selections that fail to resolve instead of rejecting the entire update. - Give each `TurnContext` a cloned `TurnEnvironmentSnapshot`, so later session configuration updates affect future turns without rewriting an active turn. - Reuse the service-owned environment manager and resolved snapshot for startup work, MCP initialization, and child-thread spawning instead of flowing resolved environments through spawn arguments. ## Test plan - `cargo check -p codex-core --tests` - `just test -p codex-core environment_selection` - `just test -p codex-core turn_environments` - `just test -p codex-core session_update_settings_does_not_rewrite_sticky_environment_cwds` - `just test -p codex-core default_turn_does_not_overlay_legacy_fallback_cwd_onto_stored_thread_environments`
pakrym-oai ·
2026-06-15 16:15:07 -07:00 -
[codex] exec-server honors remote environment cwd and shell (#28122)
## Why Next slice needed to make progress on the `remote_env_windows` test is to support passing a Windows cwd for the remote environment and using that environment's native shell. This lets the test run a real Windows process instead of only recording an early path or shell mismatch. ## What - change `TurnEnvironmentSelection.cwd` from `AbsolutePathBuf` to `PathUri` - convert local cwd values to URIs when constructing selections - preserve a remote primary cwd instead of replacing it with the local legacy fallback - prefer the selected environment's discovered shell for unified exec, falling back to the session shell when unavailable - convert back to a host-native absolute path at current native-only consumer boundaries - reject or deny unsupported foreign cwd values at the existing request-permissions boundary, with TODOs for its future migration - extend the hermetic Wine test to execute Windows PowerShell in `C:\windows` and verify successful process completion - record the current app-server rejection against the same Wine-backed remote Windows fixture when its cwd is supplied as a native Windows path
Adam Perry @ OpenAI ·
2026-06-14 06:07:46 +00:00 -
[codex] make PathUri::from_abs_path infallible (#27976)
## Why `PathUri::from_abs_path` can fail for absolute paths that do not have a normal `file:` URI representation, forcing filesystem call sites to handle a conversion error even though the original path can be preserved losslessly. ## What Make `from_abs_path` infallible and migrate its callers. Unrepresentable paths use `file:///%00/bad/path/<base64>`, encoding Unix bytes or Windows UTF-16LE; `to_abs_path` validates and decodes that fallback. The leading encoded null reserves a namespace that cannot collide with a real Unix or Windows path, and fallback URIs remain opaque to lexical path operations. ## Validation Added path-URI coverage for Unix null and non-UTF-8 paths, Windows device/verbatim and non-Unicode paths, serialization, malformed fallbacks, opaque lexical operations, invalid native payloads, and literal `/bad/path` collision resistance.
Adam Perry @ OpenAI ·
2026-06-12 16:58:42 -07:00 -
sandboxing: migrate cwd inputs to PathUri (#27816)
## Why Sandbox cwd values can cross app-server and exec-server host boundaries. They should retain URI semantics until the receiving host validates them instead of being interpreted early as native paths. ## What - Carry `PathUri` through filesystem sandbox contexts, sandbox commands, and transform inputs. - Convert command and policy cwd once in `SandboxManager::transform`, then keep launch requests native. - Preserve sandbox cwd over remote filesystem transport and reject non-native URIs without fallback. - Cache paired native/URI turn-environment cwd values during migration, with immutable access to keep them synchronized. - Extend existing protocol, forwarding, transform, and core runtime tests.
Adam Perry @ OpenAI ·
2026-06-12 11:38:01 -07:00 -
[codex] Load AGENTS.md from all bound environments (#27696)
## Why We already have the machinery to support multiple environments on a single thread, but we only show the model the contents of `AGENTS.md` files in the primary environment. We should show the model all of the relevant project instructions when we know there's more than one environment. ## Known Gaps As discussed in the RFC, this implementation: 1. doesn't handle environments being added/removed to/from the thread after its creation 2. it doesn't enforce an aggregate context budget across environments, and instead applies the configured project maximum independently to each environment ## Implementation - Discover project instructions in environment order with an independent byte budget per environment and preserve source provenance/order. - Keep the legacy fragment byte-for-byte when exactly one environment contributes project instructions; use environment-labeled sections when two or more environments contribute. - Freeze the complete rendered fragment in `LoadedAgentsMd`, insert it directly into requests, and recognize both layouts in contextual and memory filtering. - Add exact rendering, independent-budget, source-order, creation-snapshot, and consumer coverage without changing app-server schemas.
Adam Perry @ OpenAI ·
2026-06-12 00:10:06 -07:00 -
[codex] resolve environment shell metadata eagerly (#27709)
## Why Turn construction passed resolved environments through several layers while leaving the environment shell unresolved. As a result, model-visible environment context could fall back to the session shell instead of reporting the selected remote environment's shell. Resolve environment metadata at the turn-context boundary so each turn carries the shell that belongs to its selected environment. Keep request validation in app-server, where invalid selections can be returned as straightforward JSON-RPC errors without coupling core turn construction to that policy. ## What changed - resolve environment selections eagerly in `new_turn_context_from_configuration` - store the full resolved `Shell` on each `TurnEnvironment` - simplify the now-redundant resolved-environment constructor plumbing - keep duplicate and unknown-environment validation as a small app-server preflight - add a remote-environment integration test that runs a full `test_codex` turn and verifies the model-visible environment message reports `bash` ## Testing - `cargo check -p codex-core --test all -p codex-app-server` - `remote_test_env_exposes_bash_shell_to_model` on the Linux remote-executor harness
pakrym-oai ·
2026-06-11 20:35:28 -07:00 -
feat: guard git enrichment (#26175)
Skip turn git metadata enrichment when a turn has remote or multiple executors, so we do not report the orchestrator checkout as executor workspace metadata. Test: `just test -p codex-core` (blocked by existing `Session::conversation_id` compile error in `close_agent.rs`).
jif ·
2026-06-03 18:36:10 +02:00 -
Make local environment optional in EnvironmentManager (#23369)
## Summary - make `EnvironmentManager` local environment/runtime paths optional - simplify constructor surface around snapshot materialization - rename local env accessors to `require_local_environment` / `try_local_environment` ## Validation - devbox Bazel build for touched crate surfaces - `//codex-rs/exec-server:exec-server-unit-tests` - `//codex-rs/app-server-client:app-server-client-unit-tests` - filtered touched `//codex-rs/core:core-unit-tests` cases
starr-openai ·
2026-05-19 12:55:34 -07:00 -
Make environment provider snapshots path-free (#21794)
## Summary - make EnvironmentProvider::snapshot path-free and keep providers focused on provider-owned remote environments - let provider snapshots request local inclusion via include_local, with environments.toml including local and CODEX_EXEC_SERVER_URL excluding local - move reserved local environment construction into EnvironmentManager using ExecServerRuntimePaths Follow-up to https://github.com/openai/codex/pull/20667 ## Testing - just fmt - git diff --check - devbox: bazel build --bes_backend= --bes_results_url= //codex-rs/exec-server:exec-server - devbox: bazel test --bes_backend= --bes_results_url= //codex-rs/exec-server:exec-server-unit-tests Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-08 15:30:00 -07:00 -
Load configured environments from CODEX_HOME (#20667)
## Why The earlier PRs add stdio transport support and the config-backed environment provider, but the feature remains inert until normal Codex entrypoints construct `EnvironmentManager` with enough context to discover `CODEX_HOME/environments.toml`. This final stack PR activates the provider while preserving the legacy `CODEX_EXEC_SERVER_URL` fallback when no environments file exists. **Stack position:** this is PR 5 of 5. It is the product wiring PR that activates the configured environment provider added in PR 4. ## What Changed - Thread `codex_home` into `EnvironmentManagerArgs`. - Change `EnvironmentManager::new(...)` to load the provider from `CODEX_HOME`. - Preserve legacy behavior by falling back to `DefaultEnvironmentProvider::from_env()` when `environments.toml` is absent. - Make `environments.toml`-backed managers start new threads with all configured environments, default first, while keeping the legacy env-var path single-default. - Update the app-server, TUI, exec, MCP server, connector, prompt-debug, and thread-manager-sample callsites to pass `codex_home` and handle provider-loading errors. ## Self-Review Notes - The multi-environment startup path is intentionally tied to the `environments.toml` provider. Using `>1` configured environment as the only signal would also expand the legacy `CODEX_EXEC_SERVER_URL` provider because it keeps `local` addressable alongside `remote`. - The startup environment list is still derived inside `EnvironmentManager`; the provider only says whether its snapshot should start new threads with all configured environments. - The thread-manager sample was updated to pass the current `ThreadManager::new(...)` installation id argument so the stack compiles under Bazel. ## Stack - 1. https://github.com/openai/codex/pull/20663 - Add stdio exec-server listener - 2. https://github.com/openai/codex/pull/20664 - Add stdio exec-server client transport - 3. https://github.com/openai/codex/pull/20665 - Make environment providers own default selection - 4. https://github.com/openai/codex/pull/20666 - Add CODEX_HOME environments TOML provider - **5. This PR:** https://github.com/openai/codex/pull/20667 - Load configured environments from CODEX_HOME Split from original draft: https://github.com/openai/codex/pull/20508 ## Validation - `just fmt` - `git diff --check` - `bazel build --config=remote --strategy=remote --remote_download_toplevel //codex-rs/thread-manager-sample:codex-thread-manager-sample` - `bazel test --config=remote --strategy=remote --remote_download_toplevel //codex-rs/exec-server:exec-server-unit-tests` - `bazel test --config=remote --strategy=remote --remote_download_toplevel --test_sharding_strategy=disabled --test_arg=default_thread_environment_selections_use_manager_default_id //codex-rs/core:core-unit-tests` - `bazel test --config=remote --strategy=remote --remote_download_toplevel --test_sharding_strategy=disabled --test_arg=start_thread_uses_all_default_environments_from_codex_home //codex-rs/core:core-unit-tests` ## Documentation This activates `CODEX_HOME/environments.toml`; user-facing documentation should be added before this stack is treated as a documented public workflow. --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-08 11:17:56 -07:00 -
Avoid hard-coded environment context shell (#21390)
## Summary - make resolved turn environment shell metadata optional instead of hard-coding bash - render environment context shells from explicit environment metadata when present, falling back to the existing session shell - update environment context tests for inherited PowerShell-style fallback and explicit per-environment shell override ## Testing - Not run (not requested; formatted with `just fmt`). Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-06 19:54:26 +00:00 -
Prepare selected environment plumbing (#20669)
## Why This is a prep PR in the multi-environment process-tool stack. It separates ownership/config cleanup from the behavior change that teaches process tools to route by selected environment, so the follow-up PR can focus on model-facing `environment_id` behavior. ## Stack 1. https://github.com/openai/codex/pull/20646 - `EnvironmentContext` rendering for selected environments 2. https://github.com/openai/codex/pull/20669 - selected-environment ownership and tool config prep (this PR) 3. https://github.com/openai/codex/pull/20647 - process-tool `environment_id` routing ## What Changed - keep the resolved turn environment list wrapped in `ResolvedTurnEnvironments` through `TurnContext` instead of unwrapping it back to a raw `Vec` - add `TurnContext::resolve_path_against` so cwd-relative path resolution has one shared helper - replace the old tool config boolean with `ToolEnvironmentMode::{None, Single, Multiple}` ## Testing - Tests not run locally; this prep refactor is covered by GitHub CI for the stack. Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-04 17:55:49 +00:00 -
Surface multi-environment choices in environment context (#20646)
## Why The model needs a way to see which environments are available during a multi-environment turn without changing the legacy single-environment prompt surface or pulling replay/persistence changes into the same review. ## Stack 1. https://github.com/openai/codex/pull/20646 - `EnvironmentContext` rendering for selected environments (this PR) 2. https://github.com/openai/codex/pull/20669 - selected-environment ownership and tool config prep 3. https://github.com/openai/codex/pull/20647 - process-tool `environment_id` routing ## What Changed - extend `environment_context` so multi-environment turns render an `<environments>` block with the selected environment ids and cwd values - keep zero- and single-environment turns on the existing cwd-only render path - keep replay and persistence paths on the legacy surface for now so this PR stays scoped to live prompt rendering - add focused coverage in `codex-rs/core/src/context/environment_context_tests.rs` ## Testing - CI --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-01 22:11:06 +00:00 -
Use selected turn environments for runtime context (#20281)
## Summary - make selected turn environments the source of truth for session runtime cwd and MCP runtime environment selection - keep local/no-selection fallback behavior intact - add coverage for duplicate selected environments, cwd resolution, and MCP runtime environment selection ## Validation - git diff --check - rustfmt was run on touched Rust files during the implementation workflow CI should provide the full Bazel/test signal. --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-01 11:00:14 -07:00 -
Add environment provider snapshot (#20058)
## Summary - Change `EnvironmentProvider` to return concrete `Environment` instances instead of `EnvironmentConfigurations`. - Make `DefaultEnvironmentProvider` provide the provider-visible `local` environment plus optional `remote` environment from `CODEX_EXEC_SERVER_URL`. - Keep `EnvironmentManager` as the concrete cache while exposing its own explicit local environment for `local_environment()` fallback paths. ## Validation - `just fmt` - `git diff --check` --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-04-28 20:05:18 -07:00 -
Add sticky environment API and thread state (#18897)
## Summary - add sticky environment selections to app-server v2 thread/start and turn/start request flow - carry thread-level selections through core session/thread state - add app-server coverage for sticky selections and turn overrides ## Stack 1. This PR: API and thread persistence 2. #18898: config.toml named environment loading 3. #18899: downstream tool/runtime consumers ## Validation - Not run locally; split only. --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-04-23 18:57:13 -07:00