mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
[codex] rollout budget implementation (varlength 2/N) (#28494)
## Stack Depends on #28746. This PR implements shared rollout-budget accounting and model-visible reminders using the configuration defined in #28746. # Description / Main changes to Core: `AgentControl` will now be the area where "rollout level" features & accounting will have to live. It is incorrectly named for this responsibility, but I think it can hold all the necessary shared state & features (rollout token budget, mutliple thread interruption responsibilitym etc) In this PR, we have one "token ledger" that each thread will subtract from when sampling. The "charge" will occur when response.completed() is done and the calculation will be done on the responses api usage carrier. The calculation will weigh sampling and pre-fill tokens as specified. Every time the budget crosses the configured reminder threshold, a developer message is appended before the thread's next request This remaining budget will _always_ be restated/reminded after a compaction event. Expiration and fan-out interruption will be in the stacked follow-up (and also live in Agent Control). ## Reminders "You have weighted {session_tokens_left} tokens left in the shared session token budget." The first request in each thread context receives the current remainder. Later reminders are emitted after aggregate weighted usage crosses a configured interval. If several intervals are crossed before a thread sends another request, Core inserts one reminder with the latest remainder. Compaction response usage is charged before the next context starts. The next reminder is appended after the compaction summary, leaving the initial context content stable. ## Tests Integration coverage verifies: - weighted output and non-cached input accounting - initial and periodic reminders - shared accounting between a root and sub-agent - post-compaction remainder and message placement Local checks: - `just fmt` - `just test -p codex-core rollout_budget` - `git diff --check` The full workspace test suite was not run locally.
This commit is contained in:
committed by
GitHub
Unverified
parent
df5f122854
commit
32a696dbac
@@ -525,6 +525,10 @@ pub async fn thread_rollback(sess: &Arc<Session>, sub_id: String, num_turns: u32
|
||||
.collect::<Vec<_>>();
|
||||
sess.apply_rollout_reconstruction(turn_context.as_ref(), replay_items.as_slice())
|
||||
.await;
|
||||
sess.services
|
||||
.agent_control
|
||||
.rollout_budget()
|
||||
.rearm_reminder(sess.thread_id());
|
||||
sess.recompute_token_usage(turn_context.as_ref()).await;
|
||||
|
||||
sess.persist_rollout_items(&[RolloutItem::EventMsg(rollback_msg.clone())])
|
||||
|
||||
@@ -211,6 +211,7 @@ mod input_queue;
|
||||
mod mcp;
|
||||
mod multi_agents;
|
||||
mod review;
|
||||
mod rollout_budget;
|
||||
mod rollout_reconstruction;
|
||||
#[allow(clippy::module_inception)]
|
||||
pub(crate) mod session;
|
||||
@@ -3360,6 +3361,7 @@ impl Session {
|
||||
}
|
||||
state.token_info()
|
||||
};
|
||||
self.record_rollout_budget_usage(token_usage);
|
||||
if let Some(token_info) = token_info.as_ref() {
|
||||
for contributor in self.services.extensions.token_usage_contributors() {
|
||||
contributor
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
use super::session::Session;
|
||||
use super::turn_context::TurnContext;
|
||||
use crate::context::ContextualUserFragment;
|
||||
use codex_protocol::protocol::TokenUsage;
|
||||
|
||||
pub(super) async fn maybe_record_reminder(
|
||||
sess: &Session,
|
||||
turn_context: &TurnContext,
|
||||
window_id: &str,
|
||||
) {
|
||||
let budget = sess.services.agent_control.rollout_budget();
|
||||
let Some(reminder) = budget.pending_reminder(sess.thread_id(), window_id) else {
|
||||
return;
|
||||
};
|
||||
let response_item = ContextualUserFragment::into(crate::context::RolloutBudgetContext {
|
||||
remaining_tokens: reminder.remaining_tokens,
|
||||
});
|
||||
sess.record_conversation_items(turn_context, std::slice::from_ref(&response_item))
|
||||
.await;
|
||||
budget.mark_reminder_delivered(sess.thread_id(), window_id, reminder);
|
||||
}
|
||||
|
||||
impl Session {
|
||||
pub(crate) fn record_rollout_budget_usage(&self, usage: &TokenUsage) {
|
||||
self.services
|
||||
.agent_control
|
||||
.rollout_budget()
|
||||
.record_usage(usage);
|
||||
}
|
||||
}
|
||||
@@ -218,6 +218,14 @@ pub(crate) async fn run_turn(
|
||||
break;
|
||||
}
|
||||
|
||||
let window_id = sess.current_window_id().await;
|
||||
super::rollout_budget::maybe_record_reminder(
|
||||
sess.as_ref(),
|
||||
turn_context.as_ref(),
|
||||
&window_id,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Construct the input that we will send to the model.
|
||||
let sampling_request_input: Vec<ResponseItem> = async {
|
||||
sess.clone_history()
|
||||
@@ -227,7 +235,6 @@ pub(crate) async fn run_turn(
|
||||
.instrument(trace_span!("run_turn.prepare_sampling_request_input"))
|
||||
.await;
|
||||
|
||||
let window_id = sess.current_window_id().await;
|
||||
let responses_metadata = turn_context.turn_metadata_state.to_responses_metadata(
|
||||
sess.installation_id.clone(),
|
||||
window_id,
|
||||
|
||||
Reference in New Issue
Block a user