16 Commits

• core: add focused diagnostics for remote compaction overflows (#11133)

  ## Summary
  - add targeted remote-compaction failure diagnostics in compact_remote
  logging
  - log the specific values needed to explain overflow timing:
    - last_api_response_total_tokens
    - estimated_tokens_of_items_added_since_last_successful_api_response
    - estimated_bytes_of_items_added_since_last_successful_api_response
    - failing_compaction_request_body_bytes
  - simplify breakdown naming and remove
  last_api_response_total_bytes_estimate (it was an approximation and not
  useful for debugging)
  
  ## Why
  When compaction fails with context_length_exceeded, we need concrete,
  low-ambiguity numbers that map directly to:
  1) what the API most recently reported, and
  2) what local history added since then.
  
  This keeps the failure logs actionable without adding broad, noisy
  metrics.
  
  ## Testing
  - just fmt
  - cargo test -p codex-core

  Charley Cunningham · 2026-02-09 12:42:20 -08:00
  9450cd9ce5
• chore: implement Mul for TruncationPolicy (#10272)

  Codex thought this was a good idea while working on
  https://github.com/openai/codex/pull/10192.

  Michael Bolin · 2026-01-30 15:50:20 -08:00
  10ea117ee1
• Override truncation policy at model info level (#8856)

  We used to override truncation policy by comparing model info vs config
  value in context manager. A better way to do it is to construct model
  info using the config value

  Ahmed Ibrahim · 2026-01-07 13:06:20 -08:00
  187924d761
• chore: save more about turn context in rollout log file (#8458)

  ### Motivation
  - Persist richer per-turn configuration in rollouts so resumed/forked
  sessions and tooling can reason about the exact instruction inputs and
  output constraints used for a turn.
  
  ### Description
  - Extend `TurnContextItem` to include optional `base_instructions`,
  `user_instructions`, and `developer_instructions`.
  - Record the optional `final_output_json_schema` associated with a turn.
  - Add an optional `truncation_policy` to `TurnContextItem` and populate
  it when writing turn-context rollout items.
  - Introduce a protocol-level `TruncationPolicy` representation and
  convert from core truncation policy when recording.
  
  ### Testing
  - `cargo test -p codex-protocol` (pass)

  Anton Panasenko · 2025-12-22 19:51:07 -08:00
  cbc5fb9acf
• Update Model Info (#7853)
  Ahmed Ibrahim · 2025-12-11 14:06:07 -08:00
  b7fa7ca8e9
• remove model_family from `config (#7571)

  - Remove `model_family` from `config`
  - Make sure to still override config elements related to `model_family`
  like supporting reasoning

  Ahmed Ibrahim · 2025-12-04 11:57:58 -08:00
  9b2055586d
• Account for encrypted reasoning for auto compaction (#7113)

  - The total token used returned from the api doesn't account for the
  reasoning items before the assistant message
  - Account for those for auto compaction
  - Add the encrypted reasoning effort in the common tests utils
  - Add a test to make sure it works as expected

  Ahmed Ibrahim · 2025-11-22 03:06:45 +00:00
  b519267d05
• Single pass truncation (#6914)
  pakrym-oai · 2025-11-19 16:56:37 -08:00
  92e3046733
• fix: don't truncate at new lines (#6907)
  Ahmed Ibrahim · 2025-11-19 17:05:48 +00:00
  d62cab9a06
• Move shell to use truncate_text (#6842)

  Move shell to use the configurable `truncate_text`
  
  ---------
  
  Co-authored-by: pakrym-oai <pakrym@openai.com>

  Ahmed Ibrahim · 2025-11-19 01:56:08 -08:00
  efebc62fb7
• shell_command returns freeform output (#6860)

  Instead of returning structured out and then re-formatting it into
  freeform, return the freeform output from shell_command tool.
  
  Keep `shell` as the default tool for GPT-5.

  pakrym-oai · 2025-11-18 23:38:43 -08:00
  ee0484a98c
• Add the utility to truncate by tokens (#6746)

  - This PR is to make it on path for truncating by tokens. This path will
  be initially used by unified exec and context manager (responsible for
  MCP calls mainly).
  - We are exposing new config `calls_output_max_tokens`
  - Use `tokens` as the main budget unit but truncate based on the model
  family by Introducing `TruncationPolicy`.
  - Introduce `truncate_text` as a router for truncation based on the
  mode.
  
  In next PRs:
  - remove truncate_with_line_bytes_budget
  - Add the ability to the model to override the token budget.

  Ahmed Ibrahim · 2025-11-18 11:36:23 -08:00
  3de8790714
• Tighten panic on double truncation (#6701)
  Ahmed Ibrahim · 2025-11-15 07:28:59 +00:00
  3f1c4b9add
• Refactor truncation helpers into its own file (#6683)

  That's to centralize the truncation in one place. Next step would be to
  make only two methods public: one with bytes/lines and one with tokens.

  Ahmed Ibrahim · 2025-11-15 06:44:23 +00:00
  94dfb211af
• feat: use actual tokenizer for unified_exec truncation (#5514)
  jif-oai · 2025-10-23 17:08:06 +01:00
  0b4527146e
• Unified execution (#3288)

  ## Unified PTY-Based Exec Tool
  
  Note: this requires to have this flag in the config:
  `use_experimental_unified_exec_tool=true`
  
  - Adds a PTY-backed interactive exec feature (“unified_exec”) with
  session reuse via
    session_id, bounded output (128 KiB), and timeout clamping (≤ 60 s).
  - Protocol: introduces ResponseItem::UnifiedExec { session_id,
  arguments, timeout_ms }.
  - Tools: exposes unified_exec as a function tool (Responses API);
  excluded from Chat
    Completions payload while still supported in tool lists.
  - Path handling: resolves commands via PATH (or explicit paths), with
  UTF‑8/newline‑aware
    truncation (truncate_middle).
  - Tests: cover command parsing, path resolution, session
  persistence/cleanup, multi‑session
    isolation, timeouts, and truncation behavior.

  jif-oai · 2025-09-10 17:38:11 -07:00
  c09ed74a16