Share resumed rollout history (#28426)

## Summary

Resuming a persisted thread currently deep-clones its complete rollout
history several times. `InitialHistory` is retained for the app-server
response, copied into thread persistence, and copied again by read-only
accessors. These copies scale with the complete rollout rather than the
bounded model context and add measurable latency for large sessions.

This change stores resumed rollout history in `Arc<Vec<RolloutItem>>`.
Rollout loading wraps the parsed vector once, while app-server response
construction, session initialization, and thread persistence share it
through inexpensive `Arc` clones. Read-only history access now returns a
borrowed slice, and fork paths use `Arc::unwrap_or_clone` where they
genuinely need mutable ownership. Rollout reconstruction also consumes
its temporary context instead of cloning the reconstructed model
history.

The serialized representation remains unchanged. In an artificial 123 MB
rollout benchmark, sharing resumed history reduced cold resume latency
by roughly 9–10%. The affected crates compile with their test targets,
all 80 thread-store tests pass, and the Bazel dependency lock remains
valid.
This commit is contained in:
Charlie Marsh
2026-06-23 10:23:25 -04:00
committed by GitHub
parent 4147824509
commit 330ae6a516
17 changed files with 102 additions and 85 deletions
+6 -5
View File
@@ -10,6 +10,7 @@ use std::ops::Mul;
use std::path::Path;
use std::path::PathBuf;
use std::str::FromStr;
use std::sync::Arc;
use std::time::Duration;
use strum_macros::EnumIter;
@@ -2460,7 +2461,7 @@ pub struct ConversationPathResponseEvent {
#[derive(Debug, Clone, Deserialize, Serialize, JsonSchema, TS)]
pub struct ResumedHistory {
pub conversation_id: ThreadId,
pub history: Vec<RolloutItem>,
pub history: Arc<Vec<RolloutItem>>,
pub rollout_path: Option<PathBuf>,
}
@@ -2505,11 +2506,11 @@ impl InitialHistory {
}
}
pub fn get_rollout_items(&self) -> Vec<RolloutItem> {
pub fn get_rollout_items(&self) -> &[RolloutItem] {
match self {
InitialHistory::New | InitialHistory::Cleared => Vec::new(),
InitialHistory::Resumed(resumed) => resumed.history.clone(),
InitialHistory::Forked(items) => items.clone(),
InitialHistory::New | InitialHistory::Cleared => &[],
InitialHistory::Resumed(resumed) => &resumed.history,
InitialHistory::Forked(items) => items,
}
}