Commit Graph

30 Commits

  • feat(core) update Personality on turn (#9644)
    ## Summary
    Support updating Personality mid-Thread via UserTurn/OverwriteTurn. This
    is explicitly unused by the clients so far, to simplify PRs - app-server
    and tui implementations will be follow-ups.
    
    ## Testing
    - [x] added integration tests
  • Support end_turn flag (#9698)
    Experimental flag that signals the end of the turn.
  • feat(core) ModelInfo.model_instructions_template (#9597)
    ## Summary
    #9555 is the start of a rename, so I'm starting to standardize here.
    Sets up `model_instructions` templating with a strongly-typed object for
    injecting a personality block into the model instructions.
    
    ## Testing
    - [x] Added tests
    - [x] Ran locally
  • fix: prevent repeating interrupted turns (#9043)
    ## What
    Record a model-visible `<turn_aborted>` marker in history when a turn is
    interrupted, and treat it as a session prefix.
    
    ## Why
    When a turn is interrupted, Codex emits `TurnAborted` but previously did
    not persist anything model-visible in the conversation history. On the
    next user turn, the model can’t tell the previous work was aborted and
    may resume/repeat earlier actions (including duplicated side effects
    like re-opening PRs).
    
    Fixes: https://github.com/openai/codex/issues/9042
    
    ## How
    On `TurnAbortReason::Interrupted`, append a hidden user message
    containing a `<turn_aborted>…</turn_aborted>` marker and flush.
    Treat `<turn_aborted>` like `<environment_context>` for session-prefix
    filtering.
    Add a regression test to ensure follow-up turns don’t repeat side
    effects from an aborted turn.
    
    ## Testing
    `just fmt`
    `just fix -p codex-core`
    `cargo test -p codex-core -- --test-threads=1`
    `cargo test --all-features -- --test-threads=1`
    
    ---------
    
    Co-authored-by: Skylar Graika <sgraika127@gmail.com>
    Co-authored-by: jif-oai <jif@openai.com>
    Co-authored-by: Eric Traut <etraut@openai.com>
  • Act on reasoning-included per turn (#9402)
    - Reset reasoning-included flag each turn and update compaction test
  • chore(core) Create instructions module (#9422)
    ## Summary
    We have a variety of things we refer to as instructions in the code
    base: our current canonical terms are:
    - base instructions (raw string)
    - developer instructions (has a type in protocol)
    - user instructions
    
    We also have `instructions` floating around in various places. We should
    standardize on the above, and start using types to prevent them from
    ending up in the wrong place. There will be additional PRs, but I'm
    going to keep these small so we can easily follow them!
    
    ## Testing
    - [x] Tests pass, this is purely a file move
  • feat: return an error if the image sent by the user is a bad image (#9146)
    ## Before
    When we detect an `InvalidImageRequest`, we replace the image by a
    placeholder and keep going
    
    ## Now
    In such `InvalidImageRequest`, we check if the image is due to a user
    message or a tool call output. For tool call output we still replace it
    with a placeholder to avoid breaking the agentic loop bu tif this is
    because of a user message, we send an error to the user
  • remove unnecessary todos (#8842)
    > // todo(aibrahim): why are we passing model here while it can change?
    
    we update it on each turn with `.with_model`
    
    > //TODO(aibrahim): run CI in release mode.
    
    although it's good to have, release builds take double the time tests
    take.
    
    > // todo(aibrahim): make this async function
    
    we figured out another way of doing this sync
  • Merge Modelfamily into modelinfo (#8763)
    - Merge ModelFamily into ModelInfo
    - Remove logic for adding instructions to apply patch
    - Add compaction limit and visible context window to `ModelInfo`
  • chore: unify conversation with thread name (#8830)
    Done and verified by Codex + refactor feature of RustRover
  • feat(app-server): thread/rollback API (#8454)
    Add `thread/rollback` to app-server to support IDEs undo-ing the last N
    turns of a thread.
    
    For context, an IDE partner will be supporting an "undo" capability
    where the IDE (the app-server client) will be responsible for reverting
    the local changes made during the last turn. To support this well, we
    also need a way to drop the last turn (or more generally, the last N
    turns) from the agent's context. This is what `thread/rollback` does.
    
    **Core idea**: A Thread rollback is represented as a persisted event
    message (EventMsg::ThreadRollback) in the rollout JSONL file, not by
    rewriting history. On resume, both the model's context (core replay) and
    the UI turn list (app-server v2's thread history builder) apply these
    markers so the pruned history is consistent across live conversations
    and `thread/resume`.
    
    Implementation notes:
    - Rollback only affects agent context and appends to the rollout file;
    clients are responsible for reverting files on disk.
    - If a thread rollback is currently in progress, subsequent
    `thread/rollback` calls are rejected.
    - Because we use `CodexConversation::submit` and codex core tracks
    active turns, returning an error on concurrent rollbacks is communicated
    via an `EventMsg::Error` with a new variant
    `CodexErrorInfo::ThreadRollbackFailed`. app-server watches for that and
    sends the BAD_REQUEST RPC response.
    
    Tests cover thread rollbacks in both core and app-server, including when
    `num_turns` > existing turns (which clears all turns).
    
    **Note**: this explicitly does **not** behave like `/undo` which we just
    removed from the CLI, which does the opposite of what `thread/rollback`
    does. `/undo` reverts local changes via ghost commits/snapshots and does
    not modify the agent's context / conversation history.
  • Do not panic when session contains a tool call without an output (#8048)
    Normally, all tool calls within a saved session should have a response,
    but there are legitimate reasons for the response to be missing. This
    can occur if the user canceled the call or there was an error of some
    sort during the rollout. We shouldn't panic in this case.
    
    This is a partial fix for #7990
  • feat: retroactive image placeholder to prevent poisoning (#6774)
    If an image can't be read by the API, it will poison the entire history,
    preventing any new turn on the conversation.
    This detect such cases and replace the image by a placeholder
  • Account for encrypted reasoning for auto compaction (#7113)
    - The total token used returned from the api doesn't account for the
    reasoning items before the assistant message
    - Account for those for auto compaction
    - Add the encrypted reasoning effort in the common tests utils
    - Add a test to make sure it works as expected
  • Move shell to use truncate_text (#6842)
    Move shell to use the configurable `truncate_text`
    
    ---------
    
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • Add the utility to truncate by tokens (#6746)
    - This PR is to make it on path for truncating by tokens. This path will
    be initially used by unified exec and context manager (responsible for
    MCP calls mainly).
    - We are exposing new config `calls_output_max_tokens`
    - Use `tokens` as the main budget unit but truncate based on the model
    family by Introducing `TruncationPolicy`.
    - Introduce `truncate_text` as a router for truncation based on the
    mode.
    
    In next PRs:
    - remove truncate_with_line_bytes_budget
    - Add the ability to the model to override the token budget.
  • feat: remote compaction (#6795)
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • Refactor truncation helpers into its own file (#6683)
    That's to centralize the truncation in one place. Next step would be to
    make only two methods public: one with bytes/lines and one with tokens.
  • Avoid double truncation (#6631)
    1. Avoid double truncation by giving 10% above the tool default constant
    2. Add tests that fails when const = 1
  • refactor Conversation history file into its own directory (#6229)
    This is just a refactor of `conversation_history` file by breaking it up
    into multiple smaller ones with helper. This refactor will help us move
    more functionality related to context management here. in a clean way.