mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
134646eff0dfaa32d736d49dc550cde82728ea9a
121 Commits
-
core: use turn-owned world state for inline compaction (#29527)
## Why Follow-up to #29249 and its [compaction review thread](https://github.com/openai/codex/pull/29249#discussion_r3455055101). During a turn, environment readiness can change between sampling requests. Inline compaction must render the same model-visible `WorldState` used by the request it follows. Rebuilding that state during compaction can observe a newer environment, make replacement history disagree with what the model saw, and suppress the next environment update. ## What changed - Make `run_turn` own the current `Arc<WorldState>` and replace it only between sampling requests. - Build each state from an explicitly chosen environment snapshot, diff deferred-executor steps against the turn-owned state, and retain the latest state in `ContextManager` only for cross-turn and resume tracking. - Pass the exact turn-owned state into inline compaction and explicit new-context-window replacement. - Carry that state with `InitialContextInjection::BeforeLastUserMessage`, so replacement context and its stored baseline cannot come from different snapshots. - Remove obsolete state-recapture helpers and ambiguous TurnContext-only WorldState builders. - Add an integration test that moves an environment from starting to ready during a paused turn, triggers compaction, and verifies the next request receives the readiness update exactly once. ## Test plan - `just test -p codex-core deferred_executor_compaction_preserves_then_updates_environment_once` - `just test -p codex-core process_compacted_history` - `just test -p codex-core mid_turn_continuation_compaction` - `just test -p codex-core build_initial_context` - `just test -p codex-core ignores_session_prefix_messages_when_truncating`
sayan-oai ·
2026-06-23 10:33:19 -07:00 -
[codex] Use input items for Responses Lite tools (#27946)
When using Responses Lite, we should all use `additional_tools` and a developer item instead of the top level tools array & instructions field. This keeps things 1-to-1. Forced namespacing for _all_ tools will land in a following PR after some coordination & fixes in Responses API (around collisions & return items). The goal is to eventually expand the scope of this to _all_ requests from codex, but that will require larger coordination across providers & slower rollout.
rka-oai ·
2026-06-22 23:56:16 -07:00 -
[codex] migrate environment context to model world state (#29249)
## Why Environment context is model-visible state, but it is currently assembled from transient turn values and diffed through environment-specific paths. That makes initial injection, turn-to-turn updates, and changes that happen within a turn use different baselines. This PR introduces the smallest useful model world-state slice: environments only, with one in-memory baseline and one renderer for full state and diffs. ## What changed - Add a typed `WorldState` container whose sections render fragments relative to an optional previous value. Full rendering uses the same diff path with no previous state. - Replace the parallel `EnvironmentContext` representation with an `EnvironmentsState` section keyed by environment ID and rendered in deterministic order. - Preserve the legacy single-environment output while supporting multiple environments, starting environments, unavailable tombstones, and changes to persisted turn-context values. - Store the latest complete `WorldState` on `ContextManager` and use it for both turn-boundary and mid-turn environment diffs. - Build initial and post-compaction context from the same world-state builder, then retain the rendered state as the next baseline. - Seed the in-memory baseline from the latest `TurnContextItem` when resuming an existing rollout; the world state itself is not serialized. - Keep non-world settings updates on their existing path and merge rendered world-state fragments at the session consumer. ## Known limitation A legacy `TurnContextItem` only reconstructs the primary environment as `local`; it cannot faithfully recover a remote-primary environment ID after resume. Live state uses the exact environment IDs once a complete baseline is established. ## Test plan - `just test -p codex-core world_state` - `just test -p codex-core record_context_updates` - `just test -p codex-core deferred_executor_` - `just test -p codex-core build_initial_context` - `just test -p codex-core rollout_reconstruction` - `just test -p codex-core process_compacted_history_reinjects_full_initial_context`
pakrym-oai ·
2026-06-22 16:07:27 -07:00 -
core: rename metadata -> internal_chat_message_metadata_passthrough (#28968)
## Description This PR cuts Codex over from generic `ResponseItem.metadata` (introduced here: https://github.com/openai/codex/pull/28355) to `ResponseItem.internal_chat_message_metadata_passthrough`, which is the blessed path and has strongly-typed keys. For now we have to drop this MAv2 usage of `metadata`: https://github.com/openai/codex/pull/28561 until we figure out where that should live.
Owen Lin ·
2026-06-22 11:11:25 -07:00 -
core: refresh environment context before sampling (#29073)
## Why Nonblocking environment snapshots allow a turn to reach the model while a remote environment is still starting. The initial context can describe that environment as still loading, but nothing currently refreshes the model-visible environment context when startup finishes during the same turn. This adds the first request-scoped reconciliation slice on top of #28683. It is gated by `DeferredExecutor` and intentionally updates only model-visible environment context; tools and other environment-derived state will migrate separately. ## What - Add a minimal `StepContext` containing the environment snapshot captured before each sampling request. - Render attached environments with their resolved shell and starting environments with `still loading`. - Track the latest environment state recorded in model history and append a bounded update only when it changes. - Seed that baseline from full initial context so ready-at-start environments are not duplicated. - Clear the in-memory baseline when history is rewritten so replacement history can be refreshed safely. ## Testing - `just test -p codex-core deferred_executor` - `just test -p codex-core environment_context_baseline_deduplicates_until_history_is_replaced` The integration coverage verifies that a pending environment reaches the first request, the ready state reaches the next request, later requests do not duplicate it, and ready-at-start environments remain single-injected. <details> <summary>Live verification</summary> - Connected to a real remote executor with startup deliberately delayed and forced three sampling requests in one turn. - Inspected the raw model inputs: request 1 showed the remote environment as `still loading`, request 2 appended its ready shell and cwd, and request 3 contained no duplicate ready update. - With the feature disabled, startup waited for the delayed executor and the first request contained only the ready environment. - With a synchronously ready environment and the feature enabled, the first request contained one environment context with no duplicate. - Executed `pwd` and read a marker file through the remote process runner; the command exited successfully and returned the remote cwd and marker contents. </details>
sayan-oai ·
2026-06-22 08:47:43 -07:00 -
Simplify multi-agent mode controls (#29324)
## Why Multi-agent delegation policy was split across `multiAgentMode`, `features.multi_agent_mode`, and `usage_hint_enabled`. These controls could disagree: a requested mode could be downgraded by the feature flag, and disabling usage hints also disabled mode instructions. Some clients also need multi-agent tools without adding delegation-policy text to model context. The previous two-mode API could not express that directly. ## What changed `multiAgentMode` is now the only live delegation-policy control: | Mode | Behavior | | --- | --- | | `none` | Keep multi-agent tools available without adding mode instructions. | | `explicitRequestOnly` | Only delegate after an explicit user request. | | `proactive` | Delegate when parallel work materially improves speed or quality. | - new threads default to `explicitRequestOnly`; omitting the mode on later turns keeps the current value - thread start, resume, fork, and settings responses always report the concrete current mode instead of `null` - mode selection remains sticky across turns and resume - usage-hint text no longer controls whether mode instructions apply - `features.multi_agent_mode` and `usage_hint_enabled` remain accepted as ignored compatibility settings so existing configs continue to load - app-server documentation and generated schemas describe the three-mode API ## Tests - `just test -p codex-core multi_agent_mode` - `just test -p codex-core multi_agent_v2_config_from_feature_table` - `just test -p codex-core spawn_agent_description` - `just test -p codex-features` - `just test -p codex-app-server-protocol` - `just test -p codex-app-server multi_agent_mode`
jif ·
2026-06-22 10:05:36 +02:00 -
Expose thread-level multi-agent mode (#28792)
## Why Once multi-agent mode can be selected per turn, clients also need to choose the initial selection when creating a thread and observe that selection through lifecycle and settings APIs. The selected value is intentionally distinct from the effective model-visible value: no client selection is represented as `null`, even though an eligible multi-agent v2 turn derives `explicitRequestOnly` as its effective default. ## What changed - Add the optional experimental `thread/start.multiAgentMode` parameter and pass it through thread creation. - Preserve an omitted initial value as an unset selection rather than eagerly storing `explicitRequestOnly`. - Apply an explicit `thread/start` selection to the first turn through the session configuration established at thread creation. - Restore the latest persisted effective mode as the selected baseline on cold resume when rollout history contains one. - Inherit the optional selected mode from a loaded parent when creating related runtime threads. - Return the current selected `multiAgentMode` from `thread/start`, `thread/resume`, `thread/fork`, and thread settings, using `null` when no mode is selected. - Keep lifecycle reporting independent from model capability and feature eligibility; core turn construction remains responsible for calculating and persisting the effective mode. ## Not covered - Clearing an existing loaded-session selection back to unset through `turn/start`; omitted or `null` currently retains the session's selection. - A TUI control, slash command, or `config.toml` preference. ## Verification - `CARGO_INCREMENTAL=0 just test -p codex-app-server-protocol` - `CARGO_INCREMENTAL=0 just test -p codex-app-server multi_agent_mode` The focused app-server coverage verifies explicit `thread/start` initialization, first-turn prompting, nullable reporting for an omitted selection, and retention of selections that are not currently runtime-eligible. ## Stack Stacked on #28685. This PR contains only the thread initialization and lifecycle/settings API layer.
Shijie Rao ·
2026-06-19 10:50:44 +02:00 -
Add per-turn multi-agent mode (#28685)
## Why Multi-agent v2 currently carries an explicit-request-only delegation rule in its static usage hint. That provides a safe default, but it prevents clients from selecting proactive delegation per turn without changing static guidance or rewriting prior model context. This change makes delegation mode a session selection that can be updated through `turn/start`, while deriving the effective model-visible mode separately for each turn. Eligible multi-agent v2 turns remain explicit-request-only unless proactive mode is both selected and enabled. ## What changed - Add the experimental `turn/start.multiAgentMode` parameter with `explicitRequestOnly` and `proactive` values. Omission retains the loaded session's current optional selection. - Add the default-off `features.multi_agent_mode` feature gate. Eligible multi-agent v2 turns use the selected mode when enabled; an unset selection or disabled gate resolves to `explicitRequestOnly`. - Treat mode prompting as inapplicable for multi-agent v1 and other unsupported session configurations, producing no multi-agent mode developer message rather than rejecting the turn. - Move the explicit-request-only rule out of the static v2 usage hint and into a bounded, tagged developer context fragment. - Emit the effective mode in initial context and only when that effective mode changes on later turns. - Persist the effective mode in `TurnContextItem` as the durable baseline for resume and context-update comparisons. Historical rollout items are not rewritten. Later mode developer messages establish the current rule incrementally. ## Not covered - Initial selection through `thread/start` and selected-mode reporting from thread lifecycle/settings APIs; those are isolated in the stacked #28792. - A TUI control or slash command for selecting the mode. - Persisting a preferred mode to `config.toml`; selection remains session/turn scoped. - Changes to multi-agent concurrency limits, tool availability, or model catalog capability declarations. - Rewriting historical rollout prompt items. Cold resume restores the latest persisted effective mode when available while leaving historical developer messages intact. ## Verification - `CARGO_INCREMENTAL=0 just test -p codex-core multi_agent_mode` - Focused app-server coverage verifies that `turn/start.multiAgentMode` produces proactive developer instructions for an eligible v2 turn. ## Stack Followed by #28792, which adds `thread/start` initialization and lifecycle/settings observability.
Shijie Rao ·
2026-06-18 22:47:51 -07:00 -
[codex] Add optional IDs to response items (#28812)
## Why `ResponseItem` variants do not have a consistent internal ID shape: some variants carry required IDs, some carry optional IDs, and some cannot represent an ID at all. The existing fields also use inconsistent serde, TypeScript, and JSON-schema annotations. A single enum-level access path is needed before history recording can assign and retain IDs. This PR establishes that internal model only. It intentionally does not generate or serialize IDs; allocation and wire persistence are isolated in the stacked follow-up. ## What changed - Give every concrete `ResponseItem` variant an `Option<String>` ID field. - Apply the same internal-only annotations to every ID field: `#[serde(default, skip_serializing)]`, `#[ts(skip)]`, and `#[schemars(skip)]`. - Add `ResponseItem::id()` and `ResponseItem::set_id()` as the shared accessors. - Preserve IDs when history items are rewritten for truncation. - Adapt consumers that previously assumed reasoning and image-generation IDs were required. - Regenerate app-server schemas so the hidden fields are represented consistently. The serde catch-all `ResponseItem::Other` remains ID-less because it must remain a unit variant. ## Test plan - `cargo check --tests -p codex-core -p codex-api -p codex-rollout-trace -p codex-image-generation-extension` - `just test -p codex-protocol` - `just test -p codex-app-server-protocol` - `just test -p codex-api -p codex-rollout-trace -p codex-image-generation-extension` - `just test -p codex-core event_mapping`
pakrym-oai ·
2026-06-17 18:27:43 -07:00 -
Add join key for MAv2 inter-agent messages (#28561)
## Summary This keeps inter-agent communication on the existing raw response item path and adds a join key for MAv2 tool calls. MAv2 `spawn_agent`, `send_message`, and `followup_task` now stamp the originating tool call id into `ResponseItemMetadata.source_call_id` on the raw `ResponseItem::AgentMessage`. App-server clients can join that raw item back to the existing tool/activity event by call id, while using the raw agent message's existing sender, receiver, and content fields. No new app-server `ThreadItem` or notification type is added. ## Tests - `just fmt` - `just write-app-server-schema` - `just test -p codex-protocol` - `just test -p codex-app-server-protocol` - `just test -p codex-core multi_agent_v2_spawn_returns_path_and_send_message_accepts_relative_path` - `just test -p codex-core multi_agent_v2_followup_task_completion_notifies_parent_on_every_turn` - `just fix -p codex-protocol` - `just fix -p codex-app-server-protocol` - `just fix -p codex-core`
jif ·
2026-06-17 14:48:56 +02:00 -
core: remove redundant TurnContext and Prompt fields (#28638)
## Why `TurnContext` had accumulated dead fields and cached projections of values already owned by its per-turn `Config` or `ModelInfo`. Keeping both copies made ownership unclear and allowed artificial split-brain states, such as a compatibility hash differing from the model metadata it came from. `Prompt` similarly carried a write-only personality after personality selection had already been materialized into its base instructions. This makes the canonical owner explicit: configuration-backed values come from `config`, model-derived values come from `model_info`, and prompts contain only data consumed by request construction. ## What changed - Remove the unused `ghost_snapshot`, `codex_self_exe`, and `thread_source` fields. - Remove duplicate `comp_hash`, `truncation_policy`, `features`, `shell_environment_policy`, `codex_linux_sandbox_exe`, `compact_prompt`, and `tool_mode` fields. - Read those values directly from `TurnContext::config` or `TurnContext::model_info` at their consumers. - Remove the write-only `Prompt::personality` field and its constructor assignments. - Preserve review-turn inheritance of the parent turn's shell policy, Linux sandbox executable, and compact prompt through the review config. ## Testing - `cargo check -p codex-core --tests`
pakrym-oai ·
2026-06-16 22:17:24 -07:00 -
[codex] core: restore absolute turn context cwd (#28629)
## Why #28152 jumped the gun on moving the rollout format to store URIs, and would likely break compat with some features that don't go through the same types as the core logic. ## What Make `TurnContextItem.cwd` an `AbsolutePathBuf` again, remove test added for `PathUri` serialization in rollouts. Also drops a bunch of error paths that are no longer needed.
Adam Perry @ OpenAI ·
2026-06-16 19:05:26 -07:00 -
core: render remote environment cwd natively (#28152)
## Why Model-visible `<environment_context>` should match the environment of the executor, not of the app server. Stacked on #28146. ## What - Keep selected environment cwd values as `PathUri` while building environment context. - Render cwd text using the path convention represented by the URI, with the canonical URI as a fallback. - Preserve compatibility with legacy `TurnContextItem.cwd` values when reconstructing and diffing context. - Extend the Wine-backed remote Windows test to assert that the model sees `powershell` and `C:\windows`.
Adam Perry @ OpenAI ·
2026-06-16 16:17:47 -07:00 -
feat(core): add metadata field to ResponseItem (#28355)
## Description This PR adds an optional `metadata` field to `ResponseItem` for Responses API calls. Only mechanical plumbing, no actual values populated and sent yet. Turns out just adding a new field to `ResponseItem` has quite a large blast radius already. This change is backwards compatible because `metadata` is optional and omitted when absent, so existing response items and rollout history without it still deserialize and requests that do not set it keep the same wire shape. For provider compatibility, we strip out `metadata` before non-OpenAI Responses requests so Azure and AWS Bedrock never see this field. My followup PR here will actually make use of it to start storing and passing along `turn_id`: https://github.com/openai/codex/pull/28360 ## What changed - Added `ResponseItemMetadata` with optional `turn_id`, plus optional `metadata` on Responses API item variants and inter-agent communication. - Preserved item metadata through response-item rewrites such as truncation, missing tool-output synthesis, compaction history rebuilding, visible-history conversion, rollout/resume, and generated app-server schemas/types. - Strip item metadata from non-OpenAI Responses requests while preserving it for OpenAI-shaped requests. - Updated the mechanical fixture/test construction churn required by the new optional field.
Owen Lin ·
2026-06-15 15:05:28 -07:00 -
linearize history output normalization (#28309)
## Why When we prepare the conversation history, every tool call needs a matching output. Before this change, we scanned the full history again for every call. In a tool-heavy conversation, that makes the work `O(items x calls)`, or `O(n^2)` in the worst case. ## What Scan the history once and collect the IDs of existing outputs. Then each call can check its ID with an expected `O(1)` lookup. The full normalization step is now expected `O(n)`. The output order and missing-output behavior stay the same. ## Performance Based on local rollout traces, one tool-heavy session reached roughly 17,050 transcript items with about 4,292 tool-call items. On a history of that shape, the old `calls x items` scan does about 73.2 million membership checks, while the new pass does about 21.3 thousand set inserts/lookups. That is roughly 3.4k times less membership work in this normalization step. ## Validation - `just test -p codex-core normalize_` (19 passed)
jif ·
2026-06-15 18:26:34 +02:00 -
[codex] Compact when comp_hash changes (#27520)
## Summary - snapshot `comp_hash` into `TurnContext` when the turn is created and use that snapshot as the downstream source of truth - persist the turn hash in rollout context and recover it into previous-turn settings during resume and fork replay - compact existing history with the previous model only when both adjacent turns provide hashes and the values differ - record `comp_hash_changed` as the compaction reason - cover ordinary transitions, resume, and missing-hash compatibility with end-to-end tests ## Why History produced under one compaction-compatible model configuration may not be safe to carry directly into another. Compacting at the turn boundary converts that history before context updates and the new user message are added. Persisting the turn snapshot in `TurnContextItem` makes the same protection work after resuming a rollout. A missing hash is not treated as evidence of incompatibility. `None → Some`, `Some → None`, and `None → None` do not trigger compaction; only `Some(previous) → Some(current)` with unequal values does. ## Stack - depends on #27532 - #27532 is based directly on `main` ## Testing - `just test -p codex-core pre_sampling_compact_` — 6 passed - `just test -p codex-core turn_context_item_uses_turn_context_comp_hash_snapshot` — passed - `just fix -p codex-core -p codex-protocol -p codex-analytics -p codex-models-manager`
Ahmed Ibrahim ·
2026-06-11 04:11:26 +00:00 -
[codex] Remove remote compaction failure log (#27106)
## Why `log_remote_compact_failure` was the only consumer of the compact-request logging payload and most of the token-usage breakdown fields. Once that failure log is removed, keeping the surrounding carrier types leaves dead plumbing in the compaction path and context manager. ## What changed - Remove `log_remote_compact_failure`, `CompactRequestLogData`, and the v2 wrapper that only fed that log. - Let both remote compaction implementations return the original compaction error directly. - Replace `TotalTokenUsageBreakdown` with a narrow helper that returns only the remaining value needed by compaction analytics. - Keep `estimate_response_item_model_visible_bytes` private to the context manager implementation. ## Validation - `cargo check -p codex-core`
pakrym-oai ·
2026-06-08 19:23:35 -07:00 -
Encrypt multi-agent v2 message payloads (#26210)
## Why Multi-agent v2 currently routes agent instructions through normal tool arguments and inter-agent context. That means the parent model can emit plaintext task text, Codex can persist it in history/rollouts, and the recipient can receive it as ordinary assistant-message JSON. This changes the v2 path so agent instructions stay encrypted between model calls: Responses encrypts the `message` argument returned by the model, Codex forwards only that ciphertext, and Responses decrypts it internally for the recipient model. ## What changed - Mark the v2 `message` parameter as encrypted for `spawn_agent`, `send_message`, and `followup_task`. - Treat multi-agent v2 tool `message` values as ciphertext unconditionally. - Store v2 inter-agent task text in `InterAgentCommunication.encrypted_content` with empty plaintext `content`. - Convert encrypted inter-agent communications into the Responses `agent_message` input item before sending the child request. - Preserve `agent_message` items across history, rollout, compaction, telemetry, and app-server schema paths. - Leave multi-agent v1 unchanged. ## Message shape The model still calls the v2 tools with a `message` argument, but that value is now ciphertext: ```json { "name": "spawn_agent", "arguments": { "task_name": "worker", "message": "<ciphertext>" } } ``` Codex stores the task as encrypted inter-agent communication: ```json { "author": "/root", "recipient": "/root/worker", "content": "", "encrypted_content": "<ciphertext>", "trigger_turn": true } ``` When Codex builds the recipient request, it forwards the ciphertext using the new Responses input item: ```json { "type": "agent_message", "author": "/root", "recipient": "/root/worker", "content": [ { "type": "encrypted_content", "encrypted_content": "<ciphertext>" } ] } ``` Responses decrypts that item internally for the recipient model. ## Context impact - Parent context no longer carries plaintext v2 agent task instructions from these tool arguments. - Codex rollout/history stores ciphertext for v2 agent instructions. - Recipient requests receive an `agent_message` item instead of assistant commentary JSON for encrypted task delivery. - Plaintext completion/status notifications are still plaintext because they are Codex-generated status messages, not encrypted model tool arguments. ## Validation - `just test -p codex-tools` - `just test -p codex-protocol` - `just test -p codex-rollout` - `just test -p codex-rollout-trace` - `just test -p codex-otel` - `just write-app-server-schema`jif ·
2026-06-05 10:25:57 +02:00 -
Rewrite oversized tool outputs during remote compaction (#26251)
## Why When trying to fit history under compaction limit rewrite output items instead of removing them entirely. Otherwise we're breaking incrementality in relation to the previous response.
pakrym-oai ·
2026-06-03 15:25:50 -07:00 -
Add multi-agent runtime metadata types (#25720)
Stack split from #25708. Original PR intentionally left open. This first PR adds the multi-agent runtime metadata types and catalog plumbing used by the rest of the stack.
jif-oai ·
2026-06-02 12:10:14 +02:00 -
Surface filesystem permission profiles in prompt context (#23924)
## Summary Some permission profiles can encode filesystem reads that should remain unavailable to the agent. Before this change, the model-visible context and automatic approval review prompt summarized the effective permissions as a legacy sandbox mode, which can omit permission-profile filesystem entries from escalation decisions. For example, a profile can grant workspace access while denying a private subtree across every workspace root: ```toml default_permissions = "restricted-workspace" [permissions.restricted-workspace.workspace_roots] "/Users/alice/project" = true "/Users/alice/other-project" = true [permissions.restricted-workspace.filesystem] ":minimal" = "read" [permissions.restricted-workspace.filesystem.":workspace_roots"] "." = "write" "private" = "deny" "private/**" = "deny" ``` The context window now describes the workspace roots and effective filesystem side of the `PermissionProfile` directly, with deny entries marked as non-escalatable: ```xml <environment_context> <cwd>/Users/alice/project</cwd> <shell>zsh</shell> <filesystem><workspace_roots><root>/Users/alice/project</root><root>/Users/alice/other-project</root></workspace_roots><permission_profile type="managed"><file_system type="restricted"><entry access="read"><special>:minimal</special></entry><entry access="write"><path>/Users/alice/project</path></entry><entry access="write"><path>/Users/alice/other-project</path></entry><entry access="deny" escalatable="false"><path>/Users/alice/project/private</path></entry><entry access="deny" escalatable="false"><path>/Users/alice/other-project/private</path></entry><entry access="deny" escalatable="false"><glob>/Users/alice/project/private/**</glob></entry><entry access="deny" escalatable="false"><glob>/Users/alice/other-project/private/**</glob></entry></file_system></permission_profile></filesystem> </environment_context> ``` Managed requirements can impose the same kind of deny-read restriction: ```toml [permissions.filesystem] deny_read = [ "/Users/alice/project/private", "/Users/alice/project/private/**", ] ``` The automatic approval review prompt also receives the parent turn's denied-read context, so review decisions can account for the active permission profile. ## What Changed - Render the effective filesystem profile in `<environment_context>`, including profile type, filesystem entries, workspace roots, and non-escalatable deny entries. - Persist effective `workspace_roots` in `TurnContextItem` so resumed/replayed context does not have to bind `:workspace_roots` through legacy `cwd` fallback. - Add explicit permission instructions that denied reads are policy restrictions, not escalation targets. - Pass the parent turn's denied-read context into automatic approval reviews. - Add targeted coverage for prompt rendering, workspace-root materialization, replay context, and review prompt context. - Keep the prompt-context test expectations platform-aware so the same filesystem rendering assertions pass on Unix and Windows paths. ## Testing - `just test -p codex-core context::environment_context::tests::serialize_environment_context_with_full_filesystem_profile` - `just test -p codex-core context::environment_context::tests::turn_context_item_filesystem_uses_workspace_roots_instead_of_cwd` - `just test -p codex-core context::permissions_instructions::permissions_instructions_tests::builds_permissions_from_profile_with_denied_reads` - `just fix -p codex-core` I also attempted `just test -p codex-core`; the changed prompt-context tests passed, but the full local run did not complete cleanly in this sandboxed macOS environment due unrelated user-shell `CODEX_SANDBOX*` expectations and integration-test timeouts.
Michael Bolin ·
2026-05-28 14:56:53 -07:00 -
Expose conversation history to extension tools (#23963)
## Why Extension tools that need conversation context should be able to read it from the live tool invocation instead of reaching into thread persistence themselves. ## What changed - Add a `ConversationHistory` snapshot to extension `ToolCall`s and populate it from the current raw in-memory response history. - Expose all history items at this boundary so each extension can filter and bound the subset it needs before consuming or forwarding it. - Cover the adapter and registry dispatch paths and update existing extension tests that construct `ToolCall` literals. ## Test plan - `cargo test -p codex-tools` - `cargo test -p codex-extension-api` - `cargo test -p codex-goal-extension` - `cargo test -p codex-memories-extension` - `cargo test -p codex-core passes_turn_fields_to_extension_call` - `cargo test -p codex-core extension_tool_executors_are_model_visible_and_dispatchable`
sayan-oai ·
2026-05-22 01:11:47 +00:00 -
add encryptedcontent to functioncalloutput (#23500)
add new `EncryptedContent` variant to `FunctionCallOutputContentItem` ahead of standalone websearch. we need to be able to receive and pass encrypted function call output from the new web search endpoint back to responsesapi, as we cannot expose direct search results.
sayan-oai ·
2026-05-19 23:47:48 -07:00 -
[codex] Trim unused TurnContextItem fields (#22709)
## Why `TurnContextItem` is the durable baseline used to reconstruct context diffs across resume/fork. Most of the old persisted-only fields on it are no longer read, so keeping them in rollout snapshots adds schema surface and state that can drift without affecting reconstruction. `summary` is the exception: older Codex versions require it to deserialize `turn_context` records, so keep writing a default compatibility value until that schema surface can be removed safely. ## What changed - Removed the unused persisted fields from `TurnContextItem`: trace ids, user/developer instructions, output schema, and truncation policy. - Kept `summary` with a compatibility comment and made `TurnContext::to_turn_context_item` write `ReasoningSummary::Auto` instead of live turn state. - Updated rollout/context reconstruction fixtures for the retained summary field. ## Verification - `cargo test -p codex-protocol --lib turn_context_item` - `cargo test -p codex-rollout resume_candidate_matches_cwd_reads_latest_turn_context` - `cargo test -p codex-state turn_context` - `cargo test -p codex-core --lib new_default_turn_captures_current_span_trace_id` - `cargo test -p codex-core --lib record_initial_history_resumed_turn_context_after_compaction_reestablishes_reference_context_item` - `cargo test -p codex-core --test all emits_warning_when_resumed_model_differs` - `git diff --check`
pakrym-oai ·
2026-05-18 21:54:36 +00:00 -
[codex] Use compaction_trigger item for remote compaction v2 (#22809)
## Why Remote compaction v2 was still using `context_compaction` as both the request trigger and the compacted output shape. The Responses API now has the landed contract for this flow: Codex sends a dedicated `{ "type": "compaction_trigger" }` input item, and the backend returns the standard `compaction` output item with encrypted content. This aligns the v2 path with that wire contract while preserving the existing local compacted-history post-processing behavior. ## What changed - Add `ResponseItem::CompactionTrigger` and regenerate the app-server protocol schema fixtures. - Send `compaction_trigger` from `remote_compaction_v2` instead of a payload-less `context_compaction`. - Collect exactly one backend `compaction` output item, then reuse the existing compacted-history rebuilding path. - Treat the trigger item as a transient request marker rather than model output or persisted rollout/memory content. ## Verification - `cargo test -p codex-protocol compaction_trigger` - `cargo test -p codex-core remote_compact_v2` - `cargo test -p codex-core compact_remote_v2` - `cargo test -p codex-core responses_websocket_sends_response_processed_after_remote_compaction_v2` - `just write-app-server-schema` - `cargo test -p codex-app-server-protocol schema_fixtures`jif-oai ·
2026-05-15 11:40:35 +02:00 -
Deprecate TurnContext cwd and resolve_path (#22519)
## Why `TurnContext::cwd` and `TurnContext::resolve_path` are being phased out in favor of using the selected turn environment cwd directly. Deprecating both APIs makes any new direct dependency visible while preserving the existing migration path for current callers. ## What Changed - Marked `TurnContext::cwd` and `TurnContext::resolve_path` as deprecated with guidance to use the selected turn environment cwd instead. - Added exact `#[allow(deprecated)]` suppressions at each existing direct usage site, including tests, rather than adding crate-wide suppression. - Kept the change behavior-preserving: current cwd reads, writes, and path resolution continue to use the same values. ## Verification - `just fmt` - `cargo check -p codex-core` - `cargo check -p codex-core --tests` - `git diff --check`
pakrym-oai ·
2026-05-13 11:15:25 -07:00 -
chore(config) include_collaboration_mode_instructions (#22383)
## Summary Adds include_collaboration_mode_instructions, which is a config equivalent to include_permissions_instructions for collaboration modes. Desired for situations where we want to disable this instruction from entering the context ## Testing - [x] Added unit test
Dylan Hurd ·
2026-05-12 15:50:10 -07:00 -
Avoid hard-coded environment context shell (#21390)
## Summary - make resolved turn environment shell metadata optional instead of hard-coding bash - render environment context shells from explicit environment metadata when present, falling back to the existing session shell - update environment context tests for inherited PowerShell-style fallback and explicit per-environment shell override ## Testing - Not run (not requested; formatted with `just fmt`). Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-06 19:54:26 +00:00 -
feat: add remote compaction v2 Responses client path (#20773)
## Why This adds the `remote_compaction_v2` client path so remote compaction can run through the normal Responses stream and install a `context_compaction` item that trigger a compaction. The goal is to migrate some of the compaction logic on the client side We keeps the v2 transport behind a feature flag while letting follow-up requests reuse the compacted context instead of falling back to the legacy compaction item shape. ## What changed - add `ResponseItem::ContextCompaction` and refresh the generated app-server / schema / TypeScript fixtures that expose response items on the wire - add `core/src/compact_remote_v2.rs` to send compaction through the standard streamed Responses client, require exactly one `context_compaction` output item, and install that item into compacted history - route manual compact and auto-compaction through the v2 path when `remote_compaction_v2` is enabled, while keeping the existing remote compaction path as the fallback - preserve the new item type across history retention, follow-up request construction, telemetry, rollout persistence, and rollout-trace normalization - add targeted coverage for the feature flag, `context_compaction` serialization, rollout-trace normalization, and remote-compaction follow-up behavior ## Verification - added protocol tests for `context_compaction` serialization/deserialization in `protocol/src/models.rs` - added rollout-trace coverage for `context_compaction` normalization in `rollout-trace/src/reducer/conversation_tests.rs` - added remote compaction integration coverage for v2 follow-up reuse and mixed compaction output streams in `core/tests/suite/compact_remote.rs` --------- Co-authored-by: Codex <noreply@openai.com>
jif-oai ·
2026-05-04 14:15:01 +02:00 -
Surface multi-environment choices in environment context (#20646)
## Why The model needs a way to see which environments are available during a multi-environment turn without changing the legacy single-environment prompt surface or pulling replay/persistence changes into the same review. ## Stack 1. https://github.com/openai/codex/pull/20646 - `EnvironmentContext` rendering for selected environments (this PR) 2. https://github.com/openai/codex/pull/20669 - selected-environment ownership and tool config prep 3. https://github.com/openai/codex/pull/20647 - process-tool `environment_id` routing ## What Changed - extend `environment_context` so multi-environment turns render an `<environments>` block with the selected environment ids and cwd values - keep zero- and single-environment turns on the existing cwd-only render path - keep replay and persistence paths on the legacy surface for now so this PR stays scoped to live prompt rendering - add focused coverage in `codex-rs/core/src/context/environment_context_tests.rs` ## Testing - CI --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-01 22:11:06 +00:00 -
fix(core): truncate large mcp tool outputs in rollouts (#20260)
## Why Large MCP tool call outputs can make rollout JSONL files enormous. In the session that motivated this change, the biggest JSONL records were: - `event_msg/mcp_tool_call_end` - `response_item/function_call_output` both containing the same unbounded MCP payloads - just 3 MCP tool calls that each were multi-hundred MBs 😱 This PR truncates both of those JSONL records. ## How #### For `response_item/function_call_output` Unified exec already bounds tool output before it is injected into model-facing history, which also keeps the corresponding rollout `response_item/function_call_output` records small. MCP should follow the same pattern: truncate the model-facing tool output at the tool-output boundary, while leaving code-mode/raw hook consumers alone. #### For `event_msg/mcp_tool_call_end` `McpToolCallEnd` also needs its own bounded event copy because it is the app-server/replay/UI event shape that backs `ThreadItem::McpToolCall`. Unfortunately this is _not_ downstream of the `ToolOutput` trait. ## Model behavior Model behavior is actually unchanged as a result of this PR. Before this PR, MCP output was: 1. Converted to `FunctionCallOutput`. 2. Recorded into in-memory history. 3. Truncated by `ContextManager::record_items()` before later model turns saw it. After this branch, MCP output is truncated earlier, in `McpToolOutput::response_payload()`, using the same helper. Then `ContextManager::record_items()` sees an already-truncated output and effectively has little/no additional work to do. So the model should still see the same kind of truncated function-call output. The practical difference is where truncation happens: earlier, before rollout persistence/app-server emission can see the giant payload. ## Verification - `cargo test -p codex-core mcp_tool_output` - `cargo test -p codex-core mcp_tool_call::tests::truncate_mcp_tool_result_for_event` - `cargo test -p codex-core mcp_post_tool_use_payload_uses_model_tool_name_args_and_result` - `just fmt` - `just fix -p codex-core` - `git diff --check`Owen Lin ·
2026-04-30 16:30:43 +00:00 -
Remove ghost snapshots (#19481)
## Summary - Remove `ghost_snapshot` / `GhostCommit` from the Responses API surface and generated SDK/schema artifacts. - Keep legacy config loading compatible, but make undo a no-op that reports the feature is unavailable. - Clean up core history, compaction, telemetry, rollout, and tests to stop carrying ghost snapshot items. ## Testing - Unit tests passed for `codex-protocol`, `codex-core` targeted undo and compaction flows, `codex-rollout`, and `codex-app-server-protocol`. - Regenerated config and app-server schemas plus Python SDK artifacts and verified they match the checked-in outputs.
pakrym-oai ·
2026-04-27 18:48:57 -07:00 -
Cap original-detail image token estimates (#19865)
Clamp original-detail image patch estimates to the current 10k patch budget so large images cannot inflate local context accounting without bound. Add regression coverage for an over-budget image. Fixes openai/codex#19806.
Curtis 'Fjord' Hawthorne ·
2026-04-27 12:39:24 -07:00 -
Delete unused ResponseItem::Message.end_turn (#19605)
This field is unused. Delete it.
Andrey Mishchenko ·
2026-04-26 17:18:09 -07:00 -
permissions: make runtime config profile-backed (#19606)
## Why This supersedes #19391. During stack repair, GitHub marked #19391 as merged into a temporary stack branch rather than into `main`, so the runtime-config change needed a fresh PR. `PermissionProfile` is now the canonical permissions shape after #19231 because it can distinguish `Managed`, `Disabled`, and `External` enforcement while also carrying filesystem rules that legacy `SandboxPolicy` cannot represent cleanly. Core config and session state still needed to accept profile-backed permissions without forcing every profile through the strict legacy bridge, which rejected valid runtime profiles such as direct write roots. The unrelated CI/test hardening that previously rode along with this PR has been split into #19683 so this PR stays focused on the permissions model migration. ## What Changed - Adds `Permissions.permission_profile` and `SessionConfiguration.permission_profile` as constrained runtime state, while keeping `sandbox_policy` as a legacy compatibility projection. - Introduces profile setters that keep `PermissionProfile`, split filesystem/network policies, and legacy `SandboxPolicy` projections synchronized. - Uses a compatibility projection for requirement checks and legacy consumers instead of rejecting profiles that cannot round-trip through `SandboxPolicy` exactly. - Updates config loading, config overrides, session updates, turn context plumbing, prompt permission text, sandbox tags, and exec request construction to carry profile-backed runtime permissions. - Preserves configured deny-read entries and `glob_scan_max_depth` when command/session profiles are narrowed. - Adds `PermissionProfile::read_only()` and `PermissionProfile::workspace_write()` presets that match legacy defaults. ## Verification - `cargo test -p codex-core direct_write_roots` - `cargo test -p codex-core runtime_roots_to_legacy_projection` - `cargo test -p codex-app-server requested_permissions_trust_project_uses_permission_profile_intent` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/19606). * #19395 * #19394 * #19393 * #19392 * __->__ #19606
Michael Bolin ·
2026-04-26 13:29:54 -07:00 -
rollout: persist turn permission profiles (#18281)
## Why Resume and reconstruction need to preserve the permissions that were active for each user turn. If rollouts only keep legacy sandbox fields, replay cannot faithfully represent profile-shaped overrides introduced earlier in the stack. ## What changed This records `permission_profile` on user-turn rollout events, reconstructs it through history/state extraction, and updates rollout reconstruction and related fixtures to keep the field explicit. ## Verification - `cargo test -p codex-core --test all permissions_messages -- --nocapture` - `cargo test -p codex-core --test all request_permissions -- --nocapture` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/18281). * #18288 * #18287 * #18286 * #18285 * #18284 * #18283 * #18282 * __->__ #18281
Michael Bolin ·
2026-04-22 17:00:29 -07:00 -
Split DeveloperInstructions into individual fragments. (#18813)
Split DeveloperInstructions into individual fragments.
pakrym-oai ·
2026-04-21 10:22:36 -07:00 -
Organize context fragments (#18794)
Organize context fragments under `core/context`. Implement same trait on all of them.
pakrym-oai ·
2026-04-20 22:39:17 -07:00 -
Update image outputs to default to high detail (#18386)
Do not assume the default `detail`.
pakrym-oai ·
2026-04-18 11:01:12 -07:00 -
Move codex module under session (#18249)
## Summary - rename the core codex module root to session/mod.rs without using #[path] - move the codex module directory and tests under core/src/session - remove session/mod.rs reexports so call sites use explicit child module paths ## Testing - cargo test -p codex-core --lib - cargo check -p codex-core --tests - just fmt - just fix -p codex-core - git diff --check
pakrym-oai ·
2026-04-17 16:18:53 +00:00 -
feat(permissions): add glob deny-read policy support (#15979)
## Summary - adds first-class filesystem policy entries for deny-read glob patterns - parses config such as :project_roots { "**/*.env" = "none" } into pattern entries - enforces deny-read patterns in direct read/list helpers - fails closed for sandbox execution until platform backends enforce glob patterns in #18096 - preserves split filesystem policy in turn context only when it cannot be reconstructed from legacy sandbox policy ## Stack 1. This PR - glob deny-read policy/config/direct-tool support 2. #18096 - macOS and Linux sandbox enforcement 3. #17740 - managed deny-read requirements ## Verification - just fmt - cargo check -p codex-core -p codex-sandboxing --tests --------- Co-authored-by: Codex <noreply@openai.com>viyatb-oai ·
2026-04-16 10:31:51 -07:00 -
feat(guardian): send only transcript deltas on guardian followups (#17269)
## Description We reuse a guardian thread for a given user thread when we can. However, we had always sent the full transcript history every time we made a followup review request to an existing guardian thread. This is especially bad for long guardian threads since we keep re-appending old transcript entries instead of just what has changed. The fix is to just send what's new. **Caveat**: Whenever a thread is compacted or rolled back, we fall back to sending the full transcript to guardian again since the thread's history has been modified. However in the happy path we get a nice optimization. ## Before Initial guardian review sends the full parent transcript: ``` The following is the Codex agent history whose request action you are assessing... >>> TRANSCRIPT START [1] user: Please check the repo visibility and push the docs fix if needed. [2] tool gh_repo_view call: {"repo":"openai/codex"} [3] tool gh_repo_view result: repo visibility: public [4] assistant: The repo is public; I now need approval to push the docs fix. >>> TRANSCRIPT END The Codex agent has requested the following action: >>> APPROVAL REQUEST START ... >>> APPROVAL REQUEST END ``` And a followup to the same guardian thread would send the full transcript again (including items 1-4 we already sent): ``` The following is the Codex agent history whose request action you are assessing... >>> TRANSCRIPT START [1] user: Please check the repo visibility and push the docs fix if needed. [2] tool gh_repo_view call: {"repo":"openai/codex"} [3] tool gh_repo_view result: repo visibility: public [4] assistant: The repo is public; I now need approval to push the docs fix. [5] user: Please push the second docs fix too. [6] assistant: I need approval for the second docs fix. >>> TRANSCRIPT END The Codex agent has requested the following action: >>> APPROVAL REQUEST START ... >>> APPROVAL REQUEST END ``` ## After Initial guardian review sends the full parent transcript (this is unchanged): ``` The following is the Codex agent history whose request action you are assessing... >>> TRANSCRIPT START [1] user: Please check the repo visibility and push the docs fix if needed. [2] tool gh_repo_view call: {"repo":"openai/codex"} [3] tool gh_repo_view result: repo visibility: public [4] assistant: The repo is public; I now need approval to push the docs fix. >>> TRANSCRIPT END The Codex agent has requested the following action: >>> APPROVAL REQUEST START ... >>> APPROVAL REQUEST END ``` But a followup now sends: ``` The following is the Codex agent history added since your last approval assessment. Continue the same review conversation... >>> TRANSCRIPT DELTA START [5] user: Please push the second docs fix too. [6] assistant: I need approval for the second docs fix. >>> TRANSCRIPT DELTA END The Codex agent has requested the following next action: >>> APPROVAL REQUEST START ... >>> APPROVAL REQUEST END ```Owen Lin ·
2026-04-10 07:48:44 -07:00 -
[codex] allow disabling environment context injection (#16745)
This adds an `include_environment_context` config/profile flag that defaults on, and guards both initial injection and later environment updates to allow skipping injection of `<environment_context>`.
Thibault Sottiaux ·
2026-04-03 18:06:52 -07:00 -
[codex] allow disabling prompt instruction blocks (#16735)
This PR adds root and profile config switches to omit the generated `<permissions instructions>` and `<apps_instructions>` prompt blocks while keeping both enabled by default, and it gates both the initial developer-context injection and later permissions diff injection so turning the permissions block off stays effective across turn-context overrides. Also added a prompt debug tool that can be used as `codex debug prompt-input "hello"` and dumps the constructed items list.
Thibault Sottiaux ·
2026-04-03 23:47:56 +00:00 -
chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)
## Why `argument-comment-lint` was green in CI even though the repo still had many uncommented literal arguments. The main gap was target coverage: the repo wrapper did not force Cargo to inspect test-only call sites, so examples like the `latest_session_lookup_params(true, ...)` tests in `codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path. This change cleans up the existing backlog, makes the default repo lint path cover all Cargo targets, and starts rolling that stricter CI enforcement out on the platform where it is currently validated. ## What changed - mechanically fixed existing `argument-comment-lint` violations across the `codex-rs` workspace, including tests, examples, and benches - updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and `tools/argument-comment-lint/run.sh` so non-`--fix` runs default to `--all-targets` unless the caller explicitly narrows the target set - fixed both wrappers so forwarded cargo arguments after `--` are preserved with a single separator - documented the new default behavior in `tools/argument-comment-lint/README.md` - updated `rust-ci` so the macOS lint lane keeps the plain wrapper invocation and therefore enforces `--all-targets`, while Linux and Windows temporarily pass `-- --lib --bins` That temporary CI split keeps the stricter all-targets check where it is already cleaned up, while leaving room to finish the remaining Linux- and Windows-specific target-gated cleanup before enabling `--all-targets` on those runners. The Linux and Windows failures on the intermediate revision were caused by the wrapper forwarding bug, not by additional lint findings in those lanes. ## Validation - `bash -n tools/argument-comment-lint/run.sh` - `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh` - shell-level wrapper forwarding check for `-- --lib --bins` - shell-level wrapper forwarding check for `-- --tests` - `just argument-comment-lint` - `cargo test` in `tools/argument-comment-lint` - `cargo test -p codex-terminal-detection` ## Follow-up - Clean up remaining Linux-only target-gated callsites, then switch the Linux lint lane back to the plain wrapper invocation. - Clean up remaining Windows-only target-gated callsites, then switch the Windows lint lane back to the plain wrapper invocation.
Michael Bolin ·
2026-03-27 19:00:44 -07:00 -
[codex] Defer fork context injection until first turn (#15699)
## Summary - remove the fork-startup `build_initial_context` injection - keep the reconstructed `reference_context_item` as the fork baseline until the first real turn - update fork-history tests and the request snapshot, and add a `TODO(ccunningham)` for remaining nondiffable initial-context inputs ## Why Fork startup was appending current-session initial context immediately after reconstructing the parent rollout, then the first real turn could emit context updates again. That duplicated model-visible context in the child rollout. ## Impact Forked sessions now behave like resume for context seeding: startup reconstructs history and preserves the prior baseline, and the first real turn handles any current-session context emission. --------- Co-authored-by: Codex <noreply@openai.com>
Charley Cunningham ·
2026-03-24 18:34:44 -07:00 -
Move string truncation helpers into codex-utils-string (#15572)
- move the shared byte-based middle truncation logic from `core` into `codex-utils-string` - keep token-specific truncation in `codex-core` so rollout can reuse the shared helper in the next stacked PR --------- Co-authored-by: Codex <noreply@openai.com>
Ahmed Ibrahim ·
2026-03-24 15:45:40 -07:00 -
Move git utilities into a dedicated crate (#15564)
- create `codex-git-utils` and move the shared git helpers into it with file moves preserved for diff readability - move the `GitInfo` helpers out of `core` so stacked rollout work can depend on the shared crate without carrying its own git info module --------- Co-authored-by: Ahmed Ibrahim <219906144+aibrahim-oai@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>
Ahmed Ibrahim ·
2026-03-24 13:26:23 -07:00 -
Trim pre-turn context updates during rollback (#15577)
## Summary - trim contiguous developer/contextual-user pre-turn updates when rollback cuts back to a user turn - add a focused history regression test for the trim behavior - update the rollback request-boundary snapshots to show the fixed non-duplicating context shape --------- Co-authored-by: Codex <noreply@openai.com>
Charley Cunningham ·
2026-03-24 12:43:53 -07:00 -
feat: communication pattern v2 (#15647)
See internal communication
jif-oai ·
2026-03-24 18:45:49 +00:00