25 Commits

• feat: support multiple rate limits (#11260)

  Added multi-limit support end-to-end by carrying limit_name in
  rate-limit snapshots and handling multiple buckets instead of only
  codex.
  Extended /usage client parsing to consume additional_rate_limits
  Updated TUI /status and in-memory state to store/render per-limit
  snapshots
  Extended app-server rate-limit read response: kept rate_limits and added
  rate_limits_by_name.
  Adjusted usage-limit error messaging for non-default codex limit buckets

  xl-openai · 2026-02-10 20:09:31 -08:00
  fdd0cd1de9
• feat: search_tool (#10657)

  **Why We Did This**
  - The goal is to reduce MCP tool context pollution by not exposing the
  full MCP tool list up front
  - It forces an explicit discovery step (`search_tool_bm25`) so the model
  narrows tool scope before making MCP calls, which helps relevance and
  lowers prompt/tool clutter.
  
  **What It Changed**
  - Added a new experimental feature flag `search_tool` in
  `core/src/features.rs:90` and `core/src/features.rs:430`.
  - Added config/schema support for that flag in
  `core/config.schema.json:214` and `core/config.schema.json:1235`.
  - Added BM25 dependency (`bm25`) in `Cargo.toml:129` and
  `core/Cargo.toml:23`.
  - Added new tool handler `search_tool_bm25` in
  `core/src/tools/handlers/search_tool_bm25.rs:18`.
  - Registered the handler and tool spec in
  `core/src/tools/handlers/mod.rs:11` and `core/src/tools/spec.rs:780` and
  `core/src/tools/spec.rs:1344`.
  - Extended `ToolsConfig` to carry `search_tool` enablement in
  `core/src/tools/spec.rs:32` and `core/src/tools/spec.rs:56`.
  - Injected dedicated developer instructions for tool-discovery workflow
  in `core/src/codex.rs:483` and `core/src/codex.rs:1976`, using
  `core/templates/search_tool/developer_instructions.md:1`.
  - Added session state to store one-shot selected MCP tools in
  `core/src/state/session.rs:27` and `core/src/state/session.rs:131`.
  - Added filtering so when feature is enabled, only selected MCP tools
  are exposed on the next request (then consumed) in
  `core/src/codex.rs:3800` and `core/src/codex.rs:3843`.
  - Added E2E suite coverage for
  enablement/instructions/hide-until-search/one-turn-selection in
  `core/tests/suite/search_tool.rs:72`,
  `core/tests/suite/search_tool.rs:109`,
  `core/tests/suite/search_tool.rs:147`, and
  `core/tests/suite/search_tool.rs:218`.
  - Refactored test helper utilities to support config-driven tool
  collection in `core/tests/suite/tools.rs:281`.
  
  **Net Behavioral Effect**
  - With `search_tool` **off**: existing MCP behavior (tools exposed
  normally).
  - With `search_tool` **on**: MCP tools start hidden, model must call
  `search_tool_bm25`, and only returned `selected_tools` are available for
  the next model call.

  Anton Panasenko · 2026-02-09 12:53:50 -08:00
  becc3a0424
• Move warmup to the task level (#11216)

  Instead of storing a special connection on the client level make the
  regular task responsible for establishing a normal client session and
  open a connection on it.
  
  Then when the turn is started we pass in a pre-established session.

  pakrym-oai · 2026-02-09 10:57:52 -08:00
  ccd17374cb
• fix(core,app-server) resume with different model (#10719)

  ## Summary
  When resuming with a different model, we should also append a developer
  message with the model instructions
  
  ## Testing
  - [x] Added unit tests

  Dylan Hurd · 2026-02-05 00:40:05 -08:00
  fe8b474acd
• Better handling skill depdenencies on ENV VAR. (#9017)

  An experimental flow for env var skill dependencies. Skills can now
  declare required env vars in SKILL.md; if missing, the CLI prompts the
  user to get the value, and Core will store it in memory (eventually to a
  local persistent store)
  <img width="790" height="169" alt="image"
  src="https://github.com/user-attachments/assets/cd928918-9403-43cb-a7e7-b8d59bcccd9a"
  />

  xl-openai · 2026-01-29 14:13:30 -05:00
  bdd8a7d58b
• [skills] Auto install MCP dependencies when running skils with dependency specs. (#9982)

  Auto install MCP dependencies when running skils with dependency specs.

  Matthew Zeng · 2026-01-27 19:02:45 -08:00
  3bb8e69dd3
• fix(app-server, core): defer initial context write to rollout file until first turn (#9950)

  ### Overview
  Currently calling `thread/resume` will always bump the thread's
  `updated_at` timestamp. This PR makes it the `updated_at` timestamp
  changes only if a turn is triggered.
  
  ### Additonal context
  What we typically do on resuming a thread is **always** writing “initial
  context” to the rollout file immediately. This initial context includes:
  - Developer instructions derived from sandbox/approval policy + cwd
  - Optional developer instructions (if provided)
  - Optional collaboration-mode instructions
  - Optional user instructions (if provided)
  - Environment context (cwd, shell, etc.)
  
  This PR defers writing the “initial context” to the rollout file until
  the first `turn/start`, so we don't inadvertently bump the thread's
  `updated_at` timestamp until a turn is actually triggered.
  
  This works even though both `thread/resume` and `turn/start` accept
  overrides (such as `model`, `cwd`, etc.) because the initial context is
  seeded from the effective `TurnContext` in memory, computed at
  `turn/start` time, after both sets of overrides have been applied.
  
  **NOTE**: This is a very short-lived solution until we introduce sqlite.
  Then we can remove this.

  Owen Lin · 2026-01-27 10:41:54 -08:00
  fc0fd85349
• Act on reasoning-included per turn (#9402)

  - Reset reasoning-included flag each turn and update compaction test

  Ahmed Ibrahim · 2026-01-19 11:23:25 -08:00
  b11e96fb04
• fix: taking plan type from usage endpoint instead of thru auth token (#7610)

  pull plan type from the usage endpoint, persist it in session state /
  tui state, and propagate through rate limit snapshots

  zhao-oai · 2025-12-04 23:34:13 -08:00
  b8eab7ce90
• persisting credits if new snapshot does not contain credit info (#7490)

  in response to incoming changes to responses headers where the header
  may sometimes not contain credits info (no longer forcing a credit
  check)

  zhao-oai · 2025-12-02 16:23:24 -05:00
  5ebdc9af1b
• Account for encrypted reasoning for auto compaction (#7113)

  - The total token used returned from the api doesn't account for the
  reasoning items before the assistant message
  - Account for those for auto compaction
  - Add the encrypted reasoning effort in the common tests utils
  - Add a test to make sure it works as expected

  Ahmed Ibrahim · 2025-11-22 03:06:45 +00:00
  b519267d05
• Add the utility to truncate by tokens (#6746)

  - This PR is to make it on path for truncating by tokens. This path will
  be initially used by unified exec and context manager (responsible for
  MCP calls mainly).
  - We are exposing new config `calls_output_max_tokens`
  - Use `tokens` as the main budget unit but truncate based on the model
  family by Introducing `TruncationPolicy`.
  - Introduce `truncate_text` as a router for truncation based on the
  mode.
  
  In next PRs:
  - remove truncate_with_line_bytes_budget
  - Add the ability to the model to override the token budget.

  Ahmed Ibrahim · 2025-11-18 11:36:23 -08:00
  3de8790714
• feat: proxy context left after compaction (#6597)
  jif-oai · 2025-11-13 16:54:03 +01:00
  2a417c47ac
• refactor Conversation history file into its own directory (#6229)

  This is just a refactor of `conversation_history` file by breaking it up
  into multiple smaller ones with helper. This refactor will help us move
  more functionality related to context management here. in a clean way.

  Ahmed Ibrahim · 2025-11-05 10:49:35 -08:00
  1a89f70015
• Filter out reasoning items from previous turns (#5857)

  Reduces request size and prevents 400 errors when switching between API
  orgs.
  
  Based on Responses API behavior described in
  https://cookbook.openai.com/examples/responses_api/reasoning_items#caching

  pakrym-oai · 2025-10-28 11:39:34 -07:00
  1b8f2543ac
• Moving token_info to ConversationHistory (#5581)

  I want to centralize input processing and management to
  `ConversationHistory`. This would need `ConversationHistory` to have
  access to `token_info` (i.e. preventing adding a big input to the
  history). Besides, it makes more sense to have it on
  `ConversationHistory` than `state`.

  Ahmed Ibrahim · 2025-10-23 20:30:58 -07:00
  0f4fd33ddd
• Move changing turn input functionalities to ConversationHistory (#5473)

  We are doing some ad-hoc logic while dealing with conversation history.
  Ideally, we shouldn't mutate `vec[responseitem]` manually at all and
  should depend on `ConversationHistory` for those changes.
  
  Those changes are:
  - Adding input to the history
  - Removing items from the history
  - Correcting history
  
  I am also adding some `error` logs for cases we shouldn't ideally face.
  For example, we shouldn't be missing `toolcalls` or `outputs`. We
  shouldn't hit `ContextWindowExceeded` while performing `compact`
  
  This refactor will give us granular control over our context management.

  Ahmed Ibrahim · 2025-10-22 13:08:46 -07:00
  273819aaae
• Auto compact at ~90% (#5292)

  Users now hit a window exceeded limit and they usually don't know what
  to do. This starts auto compact at ~90% of the window.

  Ahmed Ibrahim · 2025-10-20 11:29:49 -07:00
  049a61bcfc
• Create independent TurnContexts (#5308)

  The goal of this change:
  1. Unify user input and user turn implementation.
  2. Have a single place where turn/session setting overrides are applied.
  3. Have a single place where turn context is created.
  4. Create TurnContext only for actual turn and have a separate structure
  for current session settings (reuse ConfigureSession)

  pakrym-oai · 2025-10-18 17:43:08 -07:00
  2287d2afde
• Surface context window error to the client (#4675)

  In the past, we were treating `input exceeded context window` as a
  streaming error and retrying on it. Retrying on it has no point because
  it won't change the behavior. In this PR, we surface the error to the
  client without retry and also send a token count event to indicate that
  the context window is full.
  
  <img width="650" height="125" alt="image"
  src="https://github.com/user-attachments/assets/c26b1213-4c27-4bfc-90f4-51a270a3efd5"
  />

  Ahmed Ibrahim · 2025-10-05 01:40:06 +00:00
  90ef94d3b3
• chore: sandbox refactor 2 (#4653)

  Revert the revert and fix the UI issue

  jif-oai · 2025-10-03 11:17:39 +01:00
  69cb72f842
• Revert "chore: sanbox extraction" (#4626)

  Reverts openai/codex#4286

  Ahmed Ibrahim · 2025-10-02 21:09:21 +00:00
  ed5d656fa8
• chore: sanbox extraction (#4286)

  # Extract and Centralize Sandboxing
  - Goal: Improve safety and clarity by centralizing sandbox planning and
  execution.
    - Approach:
  - Add planner (ExecPlan) and backend registry (Direct/Seatbelt/Linux)
  with run_with_plan.
  - Refactor codex.rs to plan-then-execute; handle failures/escalation via
  the plan.
  - Delegate apply_patch to the codex binary and run it with an empty env
  for determinism.

  jif-oai · 2025-10-01 12:05:12 +01:00
  b8195a17e5
• ref: state - 2 (#4229)

  Extracting tasks in a module and start abstraction behind a Trait (more
  to come on this but each task will be tackled in a dedicated PR)
  The goal was to drop the ActiveTask and to have a (potentially) set of
  tasks during each turn

  jif-oai · 2025-09-26 13:49:08 +00:00
  1fc3413a46
• ref: full state refactor (#4174)

  ## Current State Observations
  - `Session` currently holds many unrelated responsibilities (history,
  approval queues, task handles, rollout recorder, shell discovery, token
  tracking, etc.), making it hard to reason about ownership and lifetimes.
  - The anonymous `State` struct inside `codex.rs` mixes session-long data
  with turn-scoped queues and approval bookkeeping.
  - Turn execution (`run_task`) relies on ad-hoc local variables that
  should conceptually belong to a per-turn state object.
  - External modules (`codex::compact`, tests) frequently poke the raw
  `Session.state` mutex, which couples them to implementation details.
  - Interrupts, approvals, and rollout persistence all have bespoke
  cleanup paths, contributing to subtle bugs when a turn is aborted
  mid-flight.
  
  ## Desired End State
  - Keep a slim `Session` object that acts as the orchestrator and façade.
  It should expose a focused API (submit, approvals, interrupts, event
  emission) without storing unrelated fields directly.
  - Introduce a `state` module that encapsulates all mutable data
  structures:
  - `SessionState`: session-persistent data (history, approved commands,
  token/rate-limit info, maybe user preferences).
  - `ActiveTurn`: metadata for the currently running turn (sub-id, task
  kind, abort handle) and an `Arc<TurnState>`.
  - `TurnState`: all turn-scoped pieces (pending inputs, approval waiters,
  diff tracker, review history, auto-compact flags, last agent message,
  outstanding tool call bookkeeping).
  - Group long-lived helpers/managers into a dedicated `SessionServices`
  struct so `Session` does not accumulate "random" fields.
  - Provide clear, lock-safe APIs so other modules never touch raw
  mutexes.
  - Ensure every turn creates/drops a `TurnState` and that
  interrupts/finishes delegate cleanup to it.

  jif-oai · 2025-09-25 12:16:06 +02:00
  250b244ab4