Commit Graph

1 Commits

  • app-server: keep the model cache warm (#28699)
    ## Why
    
    The app server is long-lived, but its shared model cache otherwise
    refreshes only when a caller needs it. Once the five-minute cache
    expires, starting a thread or calling `model/list` can wait for
    `/models` on the request path.
    
    Refresh the cache in the background before it expires so foreground
    callers normally use fresh local state.
    
    ## What changed
    
    - Start an app-server worker that refreshes models immediately and then
    every three minutes using the existing models-manager API.
    - Hold only a weak reference to the models manager between refreshes, so
    the worker does not extend its lifetime.
    - Stop scheduling refreshes when the app-server lifecycle handle is shut
    down or dropped. A refresh already in progress is allowed to finish.
    - Adjust affected app-server test fixtures to distinguish the background
    `/models` probe from the connection they are testing.
    
    The existing models-manager cache, refresh strategies, auth handling,
    ETag behavior, and concurrency semantics are unchanged.
    
    ## Testing
    
    -
    `models_refresh_worker::tests::refreshes_immediately_periodically_and_stops_when_dropped`
    -
    `suite::v2::remote_control::listen_off_honors_persisted_remote_control_enable`
    -
    `suite::v2::attestation::attestation_generate_round_trip_adds_header_to_responses_websocket_handshake`