Commit Graph

13 Commits

  • [codex] Ignore fsmonitor config in Git metadata reads (#22652)
    ## Summary
    - keep Git metadata/status subprocesses independent of repository
    `core.fsmonitor` configuration
    - preserve existing working-tree state reporting while making the helper
    behavior more predictable
    - add regression coverage for `get_has_changes` when a repository
    defines an fsmonitor command
    
    ## Validation
    - `cargo fmt --all`
    - `cargo test -p codex-core test_get_has_changes_`
    - `cargo test -p codex-git-utils`
  • Emit accepted line fingerprint analytics (#21601)
    ## Why
    
    Codex assisted-code attribution needs a client-side accepted-code source
    that does not upload raw code. This adds a hash-only analytics event
    derived from the turn diff so downstream attribution can compare
    accepted Codex lines against commit or PR diffs.
    
    ## What Changed
    
    - Parse accepted/effective added lines from the final turn diff and emit
    `codex_accepted_line_fingerprints` analytics.
    - Hash repo, path, and normalized line content before upload; raw code
    and raw diffs are not included in the event.
    - Chunk large fingerprint payloads and send accepted-line fingerprint
    events in isolated requests while preserving normal batching for other
    analytics events.
    - Canonicalize Git remote URLs before repo hashing so SSH/HTTPS GitHub
    remotes join to the same repo hash.
    - Add parser coverage for unified diff hunk lines that look like `+++`
    or `---` file headers.
    
    ## Verification
    
    - `cargo test -p codex-analytics`
    - `cargo test -p codex-git-utils canonicalize_git_remote_url`
    - `just fix -p codex-analytics`
    - `just bazel-lock-check`
    - `git diff --check`
  • Disable empty Cargo test targets (#21584)
    ## Summary
    
    `cargo test` has entails both running standard Rust tests and doctests.
    It turns out that the doctest discovery is fairly slow, and it's a cost
    you pay even for crates that don't include any doctests.
    
    This PR disables doctests with `doctest = false` for crates that lack
    any doctests.
    
    For the collection of crates below, this speeds up test execution by
    >4x.
    
    E.g., before this PR:
    
    ```
    Benchmark 1: cargo test     -p codex-utils-absolute-path     -p codex-utils-cache     -p codex-utils-cli     -p codex-utils-home-dir     -p codex-utils-output-truncation     -p codex-utils-path     -p codex-utils-string     -p codex-utils-template     -p codex-utils-elapsed     -p codex-utils-json-to-toml
      Time (mean ± σ):      1.849 s ±  4.455 s    [User: 0.752 s, System: 1.367 s]
      Range (min … max):    0.418 s … 14.529 s    10 runs
    ```
    
    And after:
    
    ```
    Benchmark 1: cargo test     -p codex-utils-absolute-path     -p codex-utils-cache     -p codex-utils-cli     -p codex-utils-home-dir     -p codex-utils-output-truncation     -p codex-utils-path     -p codex-utils-string     -p codex-utils-template     -p codex-utils-elapsed     -p codex-utils-json-to-toml
      Time (mean ± σ):     428.6 ms ±   6.9 ms    [User: 187.7 ms, System: 219.7 ms]
      Range (min … max):   418.0 ms … 436.8 ms    10 runs
    ```
    
    For a single crate, with >2x speedup, before:
    
    ```
    Benchmark 1: cargo test -p codex-utils-string
      Time (mean ± σ):     491.1 ms ±   9.0 ms    [User: 229.8 ms, System: 234.9 ms]
      Range (min … max):   480.9 ms … 512.0 ms    10 runs
    ```
    
    And after:
    
    ```
    Benchmark 1: cargo test -p codex-utils-string
      Time (mean ± σ):     213.9 ms ±   4.3 ms    [User: 112.8 ms, System: 84.0 ms]
      Range (min … max):   206.8 ms … 221.0 ms    13 runs
    ```
    
    Co-authored-by: Codex <noreply@openai.com>
  • Remove ghost snapshots (#19481)
    ## Summary
    - Remove `ghost_snapshot` / `GhostCommit` from the Responses API surface
    and generated SDK/schema artifacts.
    - Keep legacy config loading compatible, but make undo a no-op that
    reports the feature is unavailable.
    - Clean up core history, compaction, telemetry, rollout, and tests to
    stop carrying ghost snapshot items.
    
    ## Testing
    - Unit tests passed for `codex-protocol`, `codex-core` targeted undo and
    compaction flows, `codex-rollout`, and `codex-app-server-protocol`.
    - Regenerated config and app-server schemas plus Python SDK artifacts
    and verified they match the checked-in outputs.
  • Refactor exec-server filesystem API into codex-file-system (#19892)
    ## Summary
    - Extracted the shared filesystem types and `ExecutorFileSystem` trait
    into a new `codex-file-system` crate
    - Switched `codex-config` and `codex-git-utils` to depend on that crate
    instead of `codex-exec-server`
    - Kept `codex-exec-server` re-exporting the same API for existing
    callers
    
    ## Testing
    - Ran `cargo test -p codex-file-system`
    - Ran `cargo test -p codex-git-utils`
    - Ran `cargo test -p codex-config`
    - Ran `cargo test -p codex-exec-server`
    - Ran `just fix -p codex-file-system`, `just fix -p codex-git-utils`,
    `just fix -p codex-config`, `just fix -p codex-exec-server`
    - Ran `just fmt`
    - Updated and verified the Bazel module lockfile
  • feat: use git-backed workspace diffs for memory consolidation (#18982)
    ## Why
    
    This PR make the `morpheus` agent (memory phase 2) use a git diff to
    start it's consolidation. The workflow is the following:
    1. The agent acquire a lock
    2. If `.codex/memories` does not exist or is not a git root, initialize
    everything (and make a first empty commit)
    3. Update `raw_memories.md` and `rollout_summaries/` as before.
    Basically we select max N phase 1 memories based on a given policy
    4. We use git (`gix`) to get a diff between the current state of
    `.codex/memories` and the last commit.
    5. Dump the diff in `phase2_workspace_diff.md`
    6. Spawn `morpheus` and point it to `phase2_workspace_diff.md`
    7. Wait for `morpheus` to be done
    8. Re-create a new `.git` and make one single commit on it. We do this
    because we don't want to preserve history through `.git` and this is
    cheap anyway
    9. We release the lock
    On top of this, we keep the retry policies etc etc
    
    The goals of this new workflow are:
    * Better support of any memory extensions such as `chronicle`
    * Allow the user to manually edit memories and this will be considered
    by the phase 2 agent
     
    As a follow-up we will need to add support for user's edition while
    `morpheus` is running
    
    ## What Changed
    
    - Added memory workspace helpers that prepare the git baseline, compute
    the diff, write `phase2_workspace_diff.md`, and reset the baseline after
    successful consolidation.
    - Updated Phase 2 to sync current inputs into `raw_memories.md` and
    `rollout_summaries/`, prune old extension resources, skip clean
    workspaces, and run the consolidation subagent only when the workspace
    has changes.
    - Tightened Phase 2 job ownership around long-running consolidation with
    heartbeats and an ownership check before resetting the baseline.
    - Simplified the prompt and state APIs so DB watermarks are bookkeeping,
    while workspace dirtiness decides whether consolidation work exists.
    - Updated the memory pipeline README and tests for workspace diffs,
    extension-resource cleanup, pollution-driven forgetting, selection
    ranking, and baseline persistence.
    
    ## Verification
    
    - Added/updated coverage in `core/src/memories/tests.rs`,
    `core/src/memories/workspace_tests.rs`, `state/src/runtime/memories.rs`,
    and `core/tests/suite/memories.rs`.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • nit: expose lib (#18962)
    As a follow-up
  • feat: baseline lib (#18848)
    This add with 2 entry point:
    * `reset_git_repository` that takes a directory and set it as a new git
    root
    * `diff_since_latest_init` this returns the diff for a given directory
    since the last `reset_git_repository`
  • Refactor config loading to use filesystem abstraction (#18209)
    Initial pass propagating FileSystem through config loading.
  • Async config loading (#18022)
    Parts of config will come from executor. Prepare for that by making
    config loading methods async.
  • [codex] Make AbsolutePathBuf joins infallible (#16981)
    Having to check for errors every time join is called is painful and
    unnecessary.
  • chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)
    ## Why
    
    `argument-comment-lint` was green in CI even though the repo still had
    many uncommented literal arguments. The main gap was target coverage:
    the repo wrapper did not force Cargo to inspect test-only call sites, so
    examples like the `latest_session_lookup_params(true, ...)` tests in
    `codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.
    
    This change cleans up the existing backlog, makes the default repo lint
    path cover all Cargo targets, and starts rolling that stricter CI
    enforcement out on the platform where it is currently validated.
    
    ## What changed
    
    - mechanically fixed existing `argument-comment-lint` violations across
    the `codex-rs` workspace, including tests, examples, and benches
    - updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
    `tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
    `--all-targets` unless the caller explicitly narrows the target set
    - fixed both wrappers so forwarded cargo arguments after `--` are
    preserved with a single separator
    - documented the new default behavior in
    `tools/argument-comment-lint/README.md`
    - updated `rust-ci` so the macOS lint lane keeps the plain wrapper
    invocation and therefore enforces `--all-targets`, while Linux and
    Windows temporarily pass `-- --lib --bins`
    
    That temporary CI split keeps the stricter all-targets check where it is
    already cleaned up, while leaving room to finish the remaining Linux-
    and Windows-specific target-gated cleanup before enabling
    `--all-targets` on those runners. The Linux and Windows failures on the
    intermediate revision were caused by the wrapper forwarding bug, not by
    additional lint findings in those lanes.
    
    ## Validation
    
    - `bash -n tools/argument-comment-lint/run.sh`
    - `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
    - shell-level wrapper forwarding check for `-- --lib --bins`
    - shell-level wrapper forwarding check for `-- --tests`
    - `just argument-comment-lint`
    - `cargo test` in `tools/argument-comment-lint`
    - `cargo test -p codex-terminal-detection`
    
    ## Follow-up
    
    - Clean up remaining Linux-only target-gated callsites, then switch the
    Linux lint lane back to the plain wrapper invocation.
    - Clean up remaining Windows-only target-gated callsites, then switch
    the Windows lint lane back to the plain wrapper invocation.
  • Move git utilities into a dedicated crate (#15564)
    - create `codex-git-utils` and move the shared git helpers into it with
    file moves preserved for diff readability
    - move the `GitInfo` helpers out of `core` so stacked rollout work can
    depend on the shared crate without carrying its own git info module
    
    ---------
    
    Co-authored-by: Ahmed Ibrahim <219906144+aibrahim-oai@users.noreply.github.com>
    Co-authored-by: Codex <noreply@openai.com>