Commit Graph

11 Commits

  • Add auth 401 observability to client bug reports (#14611)
    CXC-392
    
      [With
      401](https://openai.sentry.io/issues/7333870443/?project=4510195390611458&query=019ce8f8-560c-7f10-a00a-c59553740674&referrer=issue-stream)
      <img width="1909" height="555" alt="401 auth tags in Sentry"
      src="https://github.com/user-attachments/assets/412ea950-61c4-4780-9697-15c270971ee3"
      />
    
    
      - auth_401_*: preserved facts from the latest unauthorized response snapshot
      - auth_*: latest auth-related facts from the latest request attempt
      - auth_recovery_*: unauthorized recovery state and follow-up result
    
    
      Without 401
      <img width="1917" height="522" alt="happy-path auth tags in Sentry"
      src="https://github.com/user-attachments/assets/3381ed28-8022-43b0-b6c0-623a630e679f"
      />
    
      ###### Summary
      - Add client-visible 401 diagnostics for auth attachment, upstream auth classification, and 401 request id / cf-ray correlation.
      - Record unauthorized recovery mode, phase, outcome, and retry/follow-up status without changing auth behavior.
      - Surface the highest-signal auth and recovery fields on uploaded client bug reports so they are usable in Sentry.
      - Preserve original unauthorized evidence under `auth_401_*` while keeping follow-up result tags separate.
    
      ###### Rationale (from spec findings)
      - The dominant bucket needed proof of whether the client attached auth before send or upstream still classified the request as missing auth.
      - Client uploads needed to show whether unauthorized recovery ran and what the client tried next.
      - Request id and cf-ray needed to be preserved on the unauthorized response so server-side correlation is immediate.
      - The bug-report path needed the same auth evidence as the request telemetry path, otherwise the observability would not be operationally useful.
    
      ###### Scope
      - Add auth 401 and unauthorized-recovery observability in `codex-rs/core`, `codex-rs/codex-api`, and `codex-rs/otel`, including feedback-tag surfacing.
      - Keep auth semantics, refresh behavior, retry behavior, endpoint classification, and geo-denial follow-up work out of this PR.
    
      ###### Trade-offs
      - This exports only safe auth evidence: header presence/name, upstream auth classification, request ids, and recovery state. It does not export token values or raw upstream bodies.
      - This keeps websocket connection reuse as a transport clue because it can help distinguish stale reused sessions from fresh reconnects.
      - Misroute/base-url classification and geo-denial are intentionally deferred to a separate follow-up PR so this review stays focused on the dominant auth 401 bucket.
    
      ###### Client follow-up
      - PR 2 will add misroute/provider and geo-denial observability plus the matching feedback-tag surfacing.
      - A separate host/app-server PR should log auth-decision inputs so pre-send host auth state can be correlated with client request evidence.
      - `device_id` remains intentionally separate until there is a safe existing source on the feedback upload path.
    
      ###### Testing
      - `cargo test -p codex-core refresh_available_models_sorts_by_priority`
      - `cargo test -p codex-core emit_feedback_request_tags_`
      - `cargo test -p codex-core emit_feedback_auth_recovery_tags_`
      - `cargo test -p codex-core auth_request_telemetry_context_tracks_attached_auth_and_retry_phase`
      - `cargo test -p codex-core extract_response_debug_context_decodes_identity_headers`
      - `cargo test -p codex-core identity_auth_details`
      - `cargo test -p codex-core telemetry_error_messages_preserve_non_http_details`
      - `cargo test -p codex-core --all-features --no-run`
      - `cargo test -p codex-otel otel_export_routing_policy_routes_api_request_auth_observability`
      - `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_connect_auth_observability`
      - `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_request_transport_observability`
  • chore(otel): rename OtelManager to SessionTelemetry (#13808)
    ## Summary
    This is a purely mechanical refactor of `OtelManager` ->
    `SessionTelemetry` to better convey what the struct is doing. No
    behavior change.
    
    ## Why
    
    `OtelManager` ended up sounding much broader than what this type
    actually does. It doesn't manage OTEL globally; it's the session-scoped
    telemetry surface for emitting log/trace events and recording metrics
    with consistent session metadata (`app_version`, `model`, `slug`,
    `originator`, etc.).
    
    `SessionTelemetry` is a more accurate name, and updating the call sites
    makes that boundary a lot easier to follow.
    
    ## Validation
    
    - `just fmt`
    - `cargo test -p codex-otel`
    - `cargo test -p codex-core`
  • feat(otel, core): record turn TTFT and TTFM metrics in codex-core (#13630)
    ### Summary
    This adds turn-level latency metrics for the first model output and the
    first completed agent message.
    - `codex.turn.ttft.duration_ms` starts at turn start and records on the
    first output signal we see from the model. That includes normal
    assistant text, reasoning deltas, and non-text outputs like tool-call
    items.
    - `codex.turn.ttfm.duration_ms` also starts at turn start, but it
    records when the first agent message finishes streaming rather than when
    its first delta arrives.
    
    ### Implementation notes
    The timing is tracked in codex-core, not app-server, so the definition
    stays consistent across CLI, TUI, and app-server clients.
    
    I reused the existing turn lifecycle boundary that already drives
    `codex.turn.e2e_duration_ms`, stored the turn start timestamp in turn
    state, and record each metric once per turn.
    
    I also wired the new metric names into the OTEL runtime metrics summary
    so they show up in the same in-memory/debug snapshot path as the
    existing timing metrics.
  • Add MCP server context to otel tool_result logs (#12267)
    Summary
    - capture the origin for each configured MCP server and expose it via
    the connection manager
    - plumb MCP server name/origin into tool logging and emit
    codex.tool_result events with those fields
    - add unit coverage for origin parsing and extend OTEL tests to assert
    empty MCP fields for non-MCP tools
    - currently not logging full urls or url paths to prevent logging
    potentially sensitive data
    
    Testing
    - Not run (not requested)
  • add sandbox policy and sandbox name to codex.tool.call metrics (#10711)
    This will give visibility into the comparative success rate of the
    Windows sandbox implementations compared to other platforms.
  • fix(auth): isolate chatgptAuthTokens concept to auth manager and app-server (#10423)
    So that the rest of the codebase (like TUI) don't need to be concerned
    whether ChatGPT auth was handled by Codex itself or passed in via
    app-server's external auth mode.
  • Add websocket telemetry metrics and labels (#10316)
    Summary
    - expose websocket telemetry hooks through the responses client so
    request durations and event processing can be reported
    - record websocket request/event metrics and emit runtime telemetry
    events that the history UI now surfaces
    - improve tests to cover websocket telemetry reporting and guard runtime
    summary updates
    
    
    <img width="824" height="79" alt="Screenshot 2026-01-31 at 5 28 12 PM"
    src="https://github.com/user-attachments/assets/ea9a7965-d8b4-4e3c-a984-ef4fdc44c81d"
    />
  • feat: show runtime metrics in console (#10278)
    Summary of changes:
    
    - Adds a new feature flag: runtime_metrics
      - Declared in core/src/features.rs
      - Added to core/config.schema.json
      - Wired into OTEL init in core/src/otel_init.rs
    
    - Enables on-demand runtime metric snapshots in OTEL
      - Adds runtime_metrics: bool to otel/src/config.rs
      - Enables experimental custom reader features in otel/Cargo.toml
      - Adds snapshot/reset/summary APIs in:
        - otel/src/lib.rs
        - otel/src/metrics/client.rs
        - otel/src/metrics/config.rs
        - otel/src/metrics/error.rs
    
    - Defines metric names and a runtime summary builder
      - New files:
        - otel/src/metrics/names.rs
        - otel/src/metrics/runtime_metrics.rs
      - Summarizes totals for:
        - Tool calls
        - API requests
        - SSE/streaming events
    
    - Instruments metrics collection in OTEL manager
      - otel/src/traces/otel_manager.rs now records:
        - API call counts + durations
        - SSE event counts + durations (success/failure)
        - Tool call metrics now use shared constants
    
    - Surfaces runtime metrics in the TUI
      - Resets runtime metrics at turn start in tui/src/chatwidget.rs
    - Displays metrics in the final separator line in
    tui/src/history_cell.rs
    
    - Adds tests
      - New OTEL tests:
        - otel/tests/suite/snapshot.rs
        - otel/tests/suite/runtime_summary.rs
      - New TUI test:
    - final_message_separator_includes_runtime_metrics in
    tui/src/history_cell.rs
    
    Scope:
    - 19 files changed
    - ~652 insertions, 38 deletions
    
    
    <img width="922" height="169" alt="Screenshot 2026-01-30 at 4 11 34 PM"
    src="https://github.com/user-attachments/assets/1efd754d-a16d-4564-83a5-f4442fd2f998"
    />