mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
pakrym-oai
a4711b88dd
## Why `fs/readFile` buffers the entire file in one response, which makes large remote reads expensive and prevents callers from applying backpressure. We need an opt-in streaming path with bounded block sizes while preserving the existing single-call API for small and sandboxed reads. ## What changed - Add `ExecServerClient::stream`, returning a named `FileReadStream` that implements `futures::Stream` and yields immutable 1 MiB byte blocks. - Add internal `fs/open`, `fs/readBlock`, and `fs/close` RPCs. `fs/readBlock` accepts an explicit offset and length. - Keep unsandboxed files open between block reads, cap open handles per connection, and clean them up on EOF, error, stream drop, explicit close, or connection shutdown. - Reject platform-sandboxed streaming opens instead of turning the one-shot sandbox helper into a persistent server. Existing `fs/readFile` behavior is unchanged. ## Testing - `just test -p codex-exec-server` - Integration coverage for 1 MiB chunking, exact block-boundary EOF, sandbox rejection, and continued reads from the opened file after path replacement. - Handle-manager coverage for non-sequential offsets, variable block lengths, the 128-handle limit, and capacity release after close.
49 lines
1.4 KiB
Rust
49 lines
1.4 KiB
Rust
use std::io;
|
|
use std::path::Path;
|
|
|
|
pub(crate) async fn open(path: &Path) -> io::Result<tokio::fs::File> {
|
|
let mut options = tokio::fs::OpenOptions::new();
|
|
options.read(true);
|
|
configure_open(&mut options);
|
|
|
|
let file = options.open(path).await?;
|
|
if !is_disk_file(&file) || !file.metadata().await?.is_file() {
|
|
return Err(io::Error::new(
|
|
io::ErrorKind::InvalidInput,
|
|
format!("path `{}` is not a file", path.display()),
|
|
));
|
|
}
|
|
Ok(file)
|
|
}
|
|
|
|
#[cfg(unix)]
|
|
fn configure_open(options: &mut tokio::fs::OpenOptions) {
|
|
options.custom_flags(libc::O_NONBLOCK);
|
|
}
|
|
|
|
#[cfg(windows)]
|
|
fn configure_open(options: &mut tokio::fs::OpenOptions) {
|
|
use windows_sys::Win32::Storage::FileSystem::SECURITY_IDENTIFICATION;
|
|
|
|
options.security_qos_flags(SECURITY_IDENTIFICATION);
|
|
}
|
|
|
|
#[cfg(not(any(unix, windows)))]
|
|
fn configure_open(_options: &mut tokio::fs::OpenOptions) {}
|
|
|
|
#[cfg(windows)]
|
|
fn is_disk_file(file: &tokio::fs::File) -> bool {
|
|
use std::os::windows::io::AsRawHandle;
|
|
use windows_sys::Win32::Foundation::HANDLE;
|
|
use windows_sys::Win32::Storage::FileSystem::FILE_TYPE_DISK;
|
|
use windows_sys::Win32::Storage::FileSystem::GetFileType;
|
|
|
|
// SAFETY: `file` owns this handle for the duration of the call.
|
|
unsafe { GetFileType(file.as_raw_handle() as HANDLE) == FILE_TYPE_DISK }
|
|
}
|
|
|
|
#[cfg(not(windows))]
|
|
fn is_disk_file(_file: &tokio::fs::File) -> bool {
|
|
true
|
|
}
|