18 Commits

• core: make AGENTS.md react to environment changes (#29810)

  ## Why
  
  With deferred executors, a turn can begin before a remote environment
  attaches. AGENTS.md discovery previously ran only during session setup,
  so instructions from a later environment never reached the model or the
  session instruction sources.
  
  WorldState persistence has now landed, so this uses the durable
  model-visible baseline directly instead of carrying a temporary
  resume/fork compatibility path.
  
  ## What
  
  - Add an `AgentsMdManager` in `SessionServices` to own host
  instructions, loaded state, and refresh caching.
  - When `DeferredExecutor` is enabled, refresh AGENTS.md when attached
  environment selections change and freeze the result in the corresponding
  `StepContext`.
  - Represent AGENTS.md as a persisted WorldState section for every
  session, with bounded initial, replacement, and removal updates.
  - Remove duplicate AGENTS.md state and rendering from
  `SessionConfiguration` and `TurnContext`.
  - Build initial context, per-request updates, and compaction context
  from the same step-scoped value.
  - On resume and fork, compare current instructions with the restored
  WorldState baseline and inject a replacement exactly once when they
  differ.
  
  Builds on #29833, #29835, and #29837.
  
  ## Tests
  
  - Covers a remote environment becoming ready mid-turn, with AGENTS.md
  appearing on the next request exactly once and updating canonical
  instruction sources.
  - Covers full, unchanged, replaced, and removed AGENTS.md WorldState
  rendering.
  - Covers changed instructions across cold resume and fork without
  duplicate reinjection.
  - Covers remote-v2 compaction retaining creation-time instructions in
  the live session and cold resume appending one replacement when the
  source changed.
  - Ran focused `codex-core` AGENTS.md, WorldState, and context-update
  test suites.

  sayan-oai · 2026-06-24 22:57:42 -07:00
  f2f80ef442
• core: use turn-owned world state for inline compaction (#29527)

  ## Why
  
  Follow-up to #29249 and its [compaction review
  thread](https://github.com/openai/codex/pull/29249#discussion_r3455055101).
  
  During a turn, environment readiness can change between sampling
  requests. Inline compaction must render the same model-visible
  `WorldState` used by the request it follows. Rebuilding that state
  during compaction can observe a newer environment, make replacement
  history disagree with what the model saw, and suppress the next
  environment update.
  
  ## What changed
  
  - Make `run_turn` own the current `Arc<WorldState>` and replace it only
  between sampling requests.
  - Build each state from an explicitly chosen environment snapshot, diff
  deferred-executor steps against the turn-owned state, and retain the
  latest state in `ContextManager` only for cross-turn and resume
  tracking.
  - Pass the exact turn-owned state into inline compaction and explicit
  new-context-window replacement.
  - Carry that state with
  `InitialContextInjection::BeforeLastUserMessage`, so replacement context
  and its stored baseline cannot come from different snapshots.
  - Remove obsolete state-recapture helpers and ambiguous TurnContext-only
  WorldState builders.
  - Add an integration test that moves an environment from starting to
  ready during a paused turn, triggers compaction, and verifies the next
  request receives the readiness update exactly once.
  
  ## Test plan
  
  - `just test -p codex-core
  deferred_executor_compaction_preserves_then_updates_environment_once`
  - `just test -p codex-core process_compacted_history`
  - `just test -p codex-core mid_turn_continuation_compaction`
  - `just test -p codex-core build_initial_context`
  - `just test -p codex-core
  ignores_session_prefix_messages_when_truncating`

  sayan-oai · 2026-06-23 10:33:19 -07:00
  d1d11cac05
• core: rename metadata -> internal_chat_message_metadata_passthrough (#28968)

  ## Description
  This PR cuts Codex over from generic `ResponseItem.metadata` (introduced
  here: https://github.com/openai/codex/pull/28355) to
  `ResponseItem.internal_chat_message_metadata_passthrough`, which is the
  blessed path and has strongly-typed keys.
  
  For now we have to drop this MAv2 usage of `metadata`:
  https://github.com/openai/codex/pull/28561 until we figure out where
  that should live.

  Owen Lin · 2026-06-22 11:11:25 -07:00
  5b95745eae
• [codex] Add optional IDs to response items (#28812)

  ## Why
  
  `ResponseItem` variants do not have a consistent internal ID shape: some
  variants carry required IDs, some carry optional IDs, and some cannot
  represent an ID at all. The existing fields also use inconsistent serde,
  TypeScript, and JSON-schema annotations. A single enum-level access path
  is needed before history recording can assign and retain IDs.
  
  This PR establishes that internal model only. It intentionally does not
  generate or serialize IDs; allocation and wire persistence are isolated
  in the stacked follow-up.
  
  ## What changed
  
  - Give every concrete `ResponseItem` variant an `Option<String>` ID
  field.
  - Apply the same internal-only annotations to every ID field:
  `#[serde(default, skip_serializing)]`, `#[ts(skip)]`, and
  `#[schemars(skip)]`.
  - Add `ResponseItem::id()` and `ResponseItem::set_id()` as the shared
  accessors.
  - Preserve IDs when history items are rewritten for truncation.
  - Adapt consumers that previously assumed reasoning and image-generation
  IDs were required.
  - Regenerate app-server schemas so the hidden fields are represented
  consistently.
  
  The serde catch-all `ResponseItem::Other` remains ID-less because it
  must remain a unit variant.
  
  ## Test plan
  
  - `cargo check --tests -p codex-core -p codex-api -p codex-rollout-trace
  -p codex-image-generation-extension`
  - `just test -p codex-protocol`
  - `just test -p codex-app-server-protocol`
  - `just test -p codex-api -p codex-rollout-trace -p
  codex-image-generation-extension`
  - `just test -p codex-core event_mapping`

  pakrym-oai · 2026-06-17 18:27:43 -07:00
  dbd2857f4b
• Add join key for MAv2 inter-agent messages (#28561)

  ## Summary
  This keeps inter-agent communication on the existing raw response item
  path and adds a join key for MAv2 tool calls.
  
  MAv2 `spawn_agent`, `send_message`, and `followup_task` now stamp the
  originating tool call id into `ResponseItemMetadata.source_call_id` on
  the raw `ResponseItem::AgentMessage`. App-server clients can join that
  raw item back to the existing tool/activity event by call id, while
  using the raw agent message's existing sender, receiver, and content
  fields.
  
  No new app-server `ThreadItem` or notification type is added.
  
  ## Tests
  - `just fmt`
  - `just write-app-server-schema`
  - `just test -p codex-protocol`
  - `just test -p codex-app-server-protocol`
  - `just test -p codex-core
  multi_agent_v2_spawn_returns_path_and_send_message_accepts_relative_path`
  - `just test -p codex-core
  multi_agent_v2_followup_task_completion_notifies_parent_on_every_turn`
  - `just fix -p codex-protocol`
  - `just fix -p codex-app-server-protocol`
  - `just fix -p codex-core`

  jif · 2026-06-17 14:48:56 +02:00
  45f603302c
• feat(core): add metadata field to ResponseItem (#28355)

  ## Description
  
  This PR adds an optional `metadata` field to `ResponseItem` for
  Responses API calls. Only mechanical plumbing, no actual values
  populated and sent yet. Turns out just adding a new field to
  `ResponseItem` has quite a large blast radius already.
  
  This change is backwards compatible because `metadata` is optional and
  omitted when absent, so existing response items and rollout history
  without it still deserialize and requests that do not set it keep the
  same wire shape. For provider compatibility, we strip out `metadata`
  before non-OpenAI Responses requests so Azure and AWS Bedrock never see
  this field.
  
  My followup PR here will actually make use of it to start storing and
  passing along `turn_id`: https://github.com/openai/codex/pull/28360
  
  ## What changed
  
  - Added `ResponseItemMetadata` with optional `turn_id`, plus optional
  `metadata` on Responses API item variants and inter-agent communication.
  - Preserved item metadata through response-item rewrites such as
  truncation, missing tool-output synthesis, compaction history
  rebuilding, visible-history conversion, rollout/resume, and generated
  app-server schemas/types.
  - Strip item metadata from non-OpenAI Responses requests while
  preserving it for OpenAI-shaped requests.
  - Updated the mechanical fixture/test construction churn required by the
  new optional field.

  Owen Lin · 2026-06-15 15:05:28 -07:00
  040dafa32d
• [codex] Compact when comp_hash changes (#27520)

  ## Summary
  - snapshot `comp_hash` into `TurnContext` when the turn is created and
  use that snapshot as the downstream source of truth
  - persist the turn hash in rollout context and recover it into
  previous-turn settings during resume and fork replay
  - compact existing history with the previous model only when both
  adjacent turns provide hashes and the values differ
  - record `comp_hash_changed` as the compaction reason
  - cover ordinary transitions, resume, and missing-hash compatibility
  with end-to-end tests
  
  ## Why
  History produced under one compaction-compatible model configuration may
  not be safe to carry directly into another. Compacting at the turn
  boundary converts that history before context updates and the new user
  message are added. Persisting the turn snapshot in `TurnContextItem`
  makes the same protection work after resuming a rollout.
  
  A missing hash is not treated as evidence of incompatibility. `None →
  Some`, `Some → None`, and `None → None` do not trigger compaction; only
  `Some(previous) → Some(current)` with unequal values does.
  
  ## Stack
  - depends on #27532
  - #27532 is based directly on `main`
  
  ## Testing
  - `just test -p codex-core pre_sampling_compact_` — 6 passed
  - `just test -p codex-core
  turn_context_item_uses_turn_context_comp_hash_snapshot` — passed
  - `just fix -p codex-core -p codex-protocol -p codex-analytics -p
  codex-models-manager`

  Ahmed Ibrahim · 2026-06-11 04:11:26 +00:00
  ba4925b3c2
• [codex] Filter legacy warning messages during compaction (#22243)

  ## Why
  
  Older sessions can contain model-warning records persisted as `user`
  messages, including the unified exec process-limit warning, the
  `apply_patch`-via-`exec_command` warning, and the model-mismatch
  high-risk cyber fallback warning. Those warnings are no longer produced
  as conversation history items, but when old sessions compact they should
  still be recognized as injected context rather than preserved as real
  user turns.
  
  ## What changed
  
  - Removed `record_model_warning` and the production paths that emitted
  these warning messages into conversation history.
  - Added `LegacyUnifiedExecProcessLimitWarning`,
  `LegacyApplyPatchExecCommandWarning`, and `LegacyModelMismatchWarning`
  contextual fragments that are used only for matching old persisted
  messages.
  - Registered the legacy fragments with contextual user message detection
  so compaction filters them through the existing fragment path.
  - Added focused compaction coverage for old warning messages being
  dropped during compacted-history processing.
  
  ## Testing
  
  - `cargo test -p codex-core warning`
  - `just fix -p codex-core`

  pakrym-oai · 2026-05-11 19:51:51 -07:00
  79c65f816c
• Revert "[codex] Harden overflow auto-compaction recovery" (#22170)

  Reverts openai/codex#22141

  Ahmed Ibrahim · 2026-05-11 19:33:15 +03:00
  69f3183a8e
• [codex] Harden overflow auto-compaction recovery (#22141)

  ## Why
  Dogfooder feedback exposed two correctness gaps in normal-loop overflow
  recovery:
  
  1. a sampling request that hit `ContextWindowExceeded` could keep
  re-entering auto-compaction indefinitely if the compacted retry still
  did not fit, and
  2. local compact-history rebuilds flattened user messages down to text,
  so an overflowing `[image, "what is this?"]` turn could be retried
  without the image after compaction.
  
  That means recovery could either fail to terminate cleanly or proceed
  with a materially weakened version of the user request.
  
  ## What changed
  - Move normal-loop `ContextWindowExceeded` handling into the sampling
  retry loop, so successful rescue compaction consumes the provider retry
  budget instead of creating an unbounded outer-turn loop.
  - Keep compacted user-history rebuilds structured:
  `collect_user_messages` now carries user `UserInput` content rather than
  flattened strings, and `build_compacted_history` reconstructs full user
  messages from that structured representation.
  - Preserve image inputs while retaining the existing text-budget
  truncation behavior for compacted user history.
  - Preserve existing compaction-task failure handling and client-session
  reset behavior while bounding repeated overflow retries.
  - Add focused regression coverage for:
    - recovery after a normal-loop overflow,
    - retry-budget exhaustion after repeated overflow,
    - local recovery preserving image + text input,
    - remote recovery preserving image + text input,
    - remote compaction v2 preserving image + text input, and
    - compaction failure still terminating cleanly.
  
  The main behavior changes are in `codex-rs/core/src/session/turn.rs` and
  `codex-rs/core/src/compact.rs`.
  
  ## Verification
  - Not run locally; relying on PR CI for this update.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-05-11 16:16:49 +00:00
  15e79f3c26
• Omit service_tier from remote /responses/compact requests under API auth (#21676)

  ## Summary
  
  API-key-auth remote compaction requests should not inherit
  `service_tier` from normal `/responses` turns. This path needs to match
  API auth expectations, while ChatGPT-auth remote compaction should keep
  reusing the shared request fields that still apply there.
  
  This change keeps the decision inline in
  `codex-rs/core/src/compact_remote.rs` only. Under API key auth, the
  classic remote `/responses/compact` path now omits `service_tier`; under
  ChatGPT auth, it keeps reusing the configured tier.
  `codex-rs/core/src/compact_remote_v2.rs` is unchanged. The remote
  compaction parity coverage and snapshots were updated to assert the
  API-key omission and preserve the ChatGPT-auth behavior.
  
  ## Testing
  
  - Updated remote compaction parity coverage in
  `codex-rs/core/tests/suite/compact_remote.rs` and the corresponding
  snapshots.

  Ahmed Ibrahim · 2026-05-08 11:15:14 +03:00
  71d80f9a14
• Delete unused ResponseItem::Message.end_turn (#19605)

  This field is unused. Delete it.

  Andrey Mishchenko · 2026-04-26 17:18:09 -07:00
  35bc6e3d01
• feat: add a built-in Amazon Bedrock model provider (#18744)

  ## Why
  
  Codex needs a first-class `amazon-bedrock` model provider so users can
  select Bedrock without copying a full provider definition into
  `config.toml`. The provider has Codex-owned defaults for the pieces that
  should stay consistent across users: the display `name`, Bedrock
  `base_url`, and `wire_api`.
  
  At the same time, users still need a way to choose the AWS credential
  profile used by their local environment. This change makes
  `amazon-bedrock` a partially modifiable built-in provider: code owns the
  provider identity and endpoint defaults, while user config can set
  `model_providers.amazon-bedrock.aws.profile`.
  
  For example:
  
  ```toml
  model_provider = "amazon-bedrock"
  
  [model_providers.amazon-bedrock.aws]
  profile = "codex-bedrock"
  ```
  
  ## What Changed
  
  - Added `amazon-bedrock` to the built-in model provider map with:
    - `name = "Amazon Bedrock"`
    - `base_url = "https://bedrock-mantle.us-east-1.api.aws/v1"`
    - `wire_api = "responses"`
  - Added AWS provider auth config with a profile-only shape:
  `model_providers.<id>.aws.profile`.
  - Kept AWS auth config restricted to `amazon-bedrock`; custom providers
  that set `aws` are rejected.
  - Allowed `model_providers.amazon-bedrock` through reserved-provider
  validation so it can act as a partial override.
  - During config loading, only `aws.profile` is copied from the
  user-provided `amazon-bedrock` entry onto the built-in provider. Other
  Bedrock provider fields remain hard-coded by the built-in definition.
  - Updated the generated config schema for the new provider AWS profile
  config.

  Celia Chen · 2026-04-21 00:54:05 +00:00
  cefcfe43b9
• Update image outputs to default to high detail (#18386)

  Do not assume the default `detail`.

  pakrym-oai · 2026-04-18 11:01:12 -07:00
  53b1570367
• Move codex module under session (#18249)

  ## Summary
  - rename the core codex module root to session/mod.rs without using
  #[path]
  - move the codex module directory and tests under core/src/session
  - remove session/mod.rs reexports so call sites use explicit child
  module paths
  
  ## Testing
  - cargo test -p codex-core --lib
  - cargo check -p codex-core --tests
  - just fmt
  - just fix -p codex-core
  - git diff --check

  pakrym-oai · 2026-04-17 16:18:53 +00:00
  71e4c6fa17
• Support remote compaction for Azure responses providers (#17958)

  Azure Responses providers were still falling back to local compaction
  because the compaction gate only checked
  `ModelProviderInfo::is_openai()`.
  
  Move the capability check onto `ModelProviderInfo` with
  `supports_remote_compaction()`, backed by the existing Azure Responses
  endpoint detection used in `codex-api`, and have `core::compact`
  delegate to that helper.
  
  Add regression coverage for:
  - OpenAI providers using remote compaction
  - Azure providers using remote compaction
  - non-OpenAI/non-Azure providers staying on the local path
  
  resolves #17773
  
  ---------
  
  Co-authored-by: Michael Bolin <mbolin@openai.com>

  Ivan Murashko · 2026-04-15 13:05:11 -07:00
  f2a4925f63
• chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)

  ## Why
  
  `argument-comment-lint` was green in CI even though the repo still had
  many uncommented literal arguments. The main gap was target coverage:
  the repo wrapper did not force Cargo to inspect test-only call sites, so
  examples like the `latest_session_lookup_params(true, ...)` tests in
  `codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.
  
  This change cleans up the existing backlog, makes the default repo lint
  path cover all Cargo targets, and starts rolling that stricter CI
  enforcement out on the platform where it is currently validated.
  
  ## What changed
  
  - mechanically fixed existing `argument-comment-lint` violations across
  the `codex-rs` workspace, including tests, examples, and benches
  - updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
  `tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
  `--all-targets` unless the caller explicitly narrows the target set
  - fixed both wrappers so forwarded cargo arguments after `--` are
  preserved with a single separator
  - documented the new default behavior in
  `tools/argument-comment-lint/README.md`
  - updated `rust-ci` so the macOS lint lane keeps the plain wrapper
  invocation and therefore enforces `--all-targets`, while Linux and
  Windows temporarily pass `-- --lib --bins`
  
  That temporary CI split keeps the stricter all-targets check where it is
  already cleaned up, while leaving room to finish the remaining Linux-
  and Windows-specific target-gated cleanup before enabling
  `--all-targets` on those runners. The Linux and Windows failures on the
  intermediate revision were caused by the wrapper forwarding bug, not by
  additional lint findings in those lanes.
  
  ## Validation
  
  - `bash -n tools/argument-comment-lint/run.sh`
  - `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
  - shell-level wrapper forwarding check for `-- --lib --bins`
  - shell-level wrapper forwarding check for `-- --tests`
  - `just argument-comment-lint`
  - `cargo test` in `tools/argument-comment-lint`
  - `cargo test -p codex-terminal-detection`
  
  ## Follow-up
  
  - Clean up remaining Linux-only target-gated callsites, then switch the
  Linux lint lane back to the plain wrapper invocation.
  - Clean up remaining Windows-only target-gated callsites, then switch
  the Windows lint lane back to the plain wrapper invocation.

  Michael Bolin · 2026-03-27 19:00:44 -07:00
  61dfe0b86c
• fix: move inline codex-rs/core unit tests into sibling files (#14444)

  ## Why
  PR #13783 moved the `codex.rs` unit tests into `codex_tests.rs`. This
  applies the same extraction pattern across the rest of `codex-rs/core`
  so the production modules stay focused on runtime code instead of large
  inline test blocks.
  
  Keeping the tests in sibling files also makes follow-up edits easier to
  review because product changes no longer have to share a file with
  hundreds or thousands of lines of test scaffolding.
  
  ## What changed
  - replaced each inline `mod tests { ... }` in `codex-rs/core/src/**`
  with a path-based module declaration
  - moved each extracted unit test module into a sibling `*_tests.rs`
  file, using `mod_tests.rs` for `mod.rs` modules
  - preserved the existing `cfg(...)` guards and module-local structure so
  the refactor remains structural rather than behavioral
  
  ## Testing
  - `cargo test -p codex-core --lib` (`1653 passed; 0 failed; 5 ignored`)
  - `just fix -p codex-core`
  - `cargo fmt --check`
  - `cargo shear`

  Michael Bolin · 2026-03-12 08:16:36 -07:00
  0c8a36676a