[codex] Add turn profiling analytics (#26484)

## Summary

Add flat profiling fields to `codex_turn_event` so analytics can explain
where turn wall-clock time is spent without changing tool execution
behavior.

The profile reports:
- time before the first sampling request
- sampling time across all attempts and follow-ups
- overhead between sampling requests
- time blocked in the post-sampling tool drain
- time after the final sampling request
- sampling request and retry counts

## Implementation

- Extend the existing turn timing state with constant-memory phase
accounting and one RAII phase guard.
- Observe sampling and the existing post-sampling drain only at turn
orchestration boundaries.
- Keep tool runtime, tool futures, response item handling, and turn
lifecycle values unchanged.
- Add the profiling fields directly to the existing analytics turn event
without changing app-server protocol or rollout persistence.
- Use the existing turn `status` to distinguish completed, failed, and
interrupted profiles.

Exact sampling/tool overlap is intentionally omitted because measuring
tool completion accurately would require hooks in the tool execution
path.

## Validation

- Add app-server end-to-end coverage for a single-sampling turn with no
blocking tool work.
- Add app-server end-to-end coverage for `request_user_input` blocking
followed by a second sampling request.
- CI is running on the PR; tests were not executed locally per
repository guidance.
This commit is contained in:
Ahmed Ibrahim
2026-06-05 11:27:10 -07:00
committed by GitHub
parent 82b15b65e2
commit 8d72fb6de9
11 changed files with 530 additions and 101 deletions
@@ -63,6 +63,8 @@ use crate::facts::SubAgentThreadStartedInput;
use crate::facts::ThreadInitializationMode;
use crate::facts::TrackEventsContext;
use crate::facts::TurnCodexErrorFact;
use crate::facts::TurnProfile;
use crate::facts::TurnProfileFact;
use crate::facts::TurnResolvedConfigFact;
use crate::facts::TurnStatus;
use crate::facts::TurnSteerRequestError;
@@ -396,6 +398,18 @@ fn sample_turn_resolved_config(thread_id: &str, turn_id: &str) -> TurnResolvedCo
}
}
fn sample_turn_profile() -> TurnProfile {
TurnProfile {
before_first_sampling_ms: 100,
sampling_ms: 700,
between_sampling_overhead_ms: 50,
tool_blocking_ms: 250,
after_last_sampling_ms: 134,
sampling_request_count: 2,
sampling_retry_count: 1,
}
}
fn sample_turn_steer_request(
thread_id: &str,
expected_turn_id: &str,
@@ -649,6 +663,18 @@ async fn ingest_turn_prerequisites(
)
.await;
}
reducer
.ingest(
AnalyticsFact::Custom(CustomAnalyticsFact::TurnProfile(Box::new(
TurnProfileFact {
turn_id: "turn-2".to_string(),
profile: sample_turn_profile(),
},
))),
out,
)
.await;
}
async fn ingest_review_prerequisites(
@@ -3300,6 +3326,13 @@ fn turn_event_serializes_expected_shape() {
output_tokens: None,
reasoning_output_tokens: None,
total_tokens: None,
before_first_sampling_ms: 100,
sampling_ms: 700,
between_sampling_overhead_ms: 50,
tool_blocking_ms: 250,
after_last_sampling_ms: 134,
sampling_request_count: 2,
sampling_retry_count: 1,
duration_ms: Some(1234),
started_at: Some(455),
completed_at: Some(456),
@@ -3366,6 +3399,13 @@ fn turn_event_serializes_expected_shape() {
"output_tokens": null,
"reasoning_output_tokens": null,
"total_tokens": null,
"before_first_sampling_ms": 100,
"sampling_ms": 700,
"between_sampling_overhead_ms": 50,
"tool_blocking_ms": 250,
"after_last_sampling_ms": 134,
"sampling_request_count": 2,
"sampling_retry_count": 1,
"duration_ms": 1234,
"started_at": 455,
"completed_at": 456