Cod1ng dc4524e960 fix: handle UTF-8 multi-byte characters split across stream chunk boundaries (#1923) ...

* fix: handle UTF-8 multi-byte characters split across stream chunk boundaries

Replace String::from_utf8_lossy with append_utf8_safe in all four SSE
streaming paths. When a multi-byte UTF-8 character (e.g. Chinese, emoji)
is split across TCP chunk boundaries, from_utf8_lossy silently replaces
the incomplete halves with U+FFFD (�). This caused intermittent garbled
output in Claude Code when using the Copilot reverse proxy, because the
format conversion streams reconstruct SSE events from the corrupted buffer.

The new append_utf8_safe function preserves incomplete trailing bytes in
a remainder buffer and merges them with the next chunk before decoding,
ensuring characters are never split during UTF-8 conversion.

Fixes: intermittent U+FFFD replacement characters in Claude Code output
via Copilot proxy (not reproducible with direct Copilot connections like
opencode because they pass through raw bytes without format conversion).

* style: fix cargo fmt formatting in UTF-8 boundary tests

---------

Co-authored-by: Cod1ng <codingts@gmail.com>
Co-authored-by: encodets <encodets@gmail.com>

dc4524e960 · 2026-04-08 10:02:35 +08:00

History

..

providers

fix: handle UTF-8 multi-byte characters split across stream chunk boundaries (#1923)

2026-04-08 10:02:35 +08:00

usage

style: format Rust code with cargo fmt

2026-03-08 22:27:53 +08:00

body_filter.rs

chore(proxy): remove unused body filter helper

2026-01-04 23:00:13 +08:00

cache_injector.rs

feat: add Bedrock request optimizer (PRE-SEND thinking + cache injection) (#1301)

2026-03-07 18:57:21 +08:00

circuit_breaker.rs

feat(proxy): add thinking signature rectifier for Claude API (#595)

2026-01-14 00:12:13 +08:00

copilot_optimizer.rs

feat: add Copilot optimizer to reduce premium interaction consumption

2026-04-05 08:34:10 +08:00

error_mapper.rs

Feat/usage improvements (#508)

2025-12-31 22:57:00 +08:00

error.rs

refactor(proxy): transparent header forwarding via hyper client (#1714)

2026-03-29 20:26:15 +08:00

failover_switch.rs

fix(proxy): serialize per-app provider switches to prevent state corruption

2026-03-31 22:46:42 +08:00

forwarder.rs

feat: add Copilot optimizer to reduce premium interaction consumption

2026-04-05 08:34:10 +08:00

handler_config.rs

Feat/usage improvements (#508)

2025-12-31 22:57:00 +08:00

handler_context.rs

feat: add Copilot optimizer to reduce premium interaction consumption

2026-04-05 08:34:10 +08:00

handlers.rs

refactor(proxy): transparent header forwarding via hyper client (#1714)

2026-03-29 20:26:15 +08:00

health.rs

Feat/proxy server (#355)

2025-12-05 11:26:41 +08:00

http_client.rs

refactor(proxy): transparent header forwarding via hyper client (#1714)

2026-03-29 20:26:15 +08:00

hyper_client.rs

fix(copilot): 修复 GitHub Copilot 认证和代理问题 (#1854)

2026-04-04 22:52:23 +08:00

log_codes.rs

refactor(proxy): transparent header forwarding via hyper client (#1714)

2026-03-29 20:26:15 +08:00

mod.rs

feat: add Copilot optimizer to reduce premium interaction consumption

2026-04-05 08:34:10 +08:00

model_mapper.rs

feat(proxy): fix thinking rectifiers and resolve clippy warnings (#1005)

2026-02-12 22:28:05 +08:00

provider_router.rs

fix(test): resolve HOME env race condition in parallel tests

2026-04-02 22:08:12 +08:00

response_handler.rs

fix: handle UTF-8 multi-byte characters split across stream chunk boundaries (#1923)

2026-04-08 10:02:35 +08:00

response_processor.rs

fix: handle UTF-8 multi-byte characters split across stream chunk boundaries (#1923)

2026-04-08 10:02:35 +08:00

server.rs

refactor(proxy): transparent header forwarding via hyper client (#1714)

2026-03-29 20:26:15 +08:00

session.rs

Feat/usage improvements (#508)

2025-12-31 22:57:00 +08:00

sse.rs

fix: handle UTF-8 multi-byte characters split across stream chunk boundaries (#1923)

2026-04-08 10:02:35 +08:00

switch_lock.rs

fix(proxy): serialize per-app provider switches to prevent state corruption

2026-03-31 22:46:42 +08:00

thinking_budget_rectifier.rs

feat(proxy): fix thinking rectifiers and resolve clippy warnings (#1005)

2026-02-12 22:28:05 +08:00

thinking_optimizer.rs

refactor(proxy): transparent header forwarding via hyper client (#1714)

2026-03-29 20:26:15 +08:00

thinking_rectifier.rs

feat(proxy): fix thinking rectifiers and resolve clippy warnings (#1005)

2026-02-12 22:28:05 +08:00

types.rs

feat: add Copilot optimizer to reduce premium interaction consumption

2026-04-05 08:34:10 +08:00