mirror of
https://github.com/router-for-me/CLIProxyAPI.git
synced 2026-02-03 04:50:52 +08:00
feat(translator): add token counting functionality for Gemini, Claude, and CLI - Introduced `TokenCount` handling across various Codex translators (Gemini, Claude, CLI) with respective implementations. - Added utility methods for token counting and formatting responses. - Integrated `tiktoken-go/tokenizer` library for tokenization. - Updated CodexExecutor with token counting logic to support multiple models including GPT-5 variants. - Refined go.mod and go.sum to include new dependencies. feat(runtime): add token counting functionality across executors - Implemented token counting in OpenAICompatExecutor, QwenExecutor, and IFlowExecutor. - Added utilities for token counting and response formatting using `tiktoken-go/tokenizer`. - Integrated token counting into translators for Gemini, Claude, and Gemini CLI. - Enhanced multiple model support, including GPT-5 variants, for token counting. docs: update environment variable instructions for multi-model support - Added details for setting `ANTHROPIC_DEFAULT_OPUS_MODEL`, `ANTHROPIC_DEFAULT_SONNET_MODEL`, and `ANTHROPIC_DEFAULT_HAIKU_MODEL` for version 2.x.x. - Clarified usage of `ANTHROPIC_MODEL` and `ANTHROPIC_SMALL_FAST_MODEL` for version 1.x.x. - Expanded examples for setting environment variables across different models including Gemini, GPT-5, Claude, and Qwen3.
This commit is contained in:
@@ -354,3 +354,7 @@ func buildReverseMapFromClaudeOriginalShortToOriginal(original []byte) map[strin
|
||||
}
|
||||
return rev
|
||||
}
|
||||
|
||||
func ClaudeTokenCount(ctx context.Context, count int64) string {
|
||||
return fmt.Sprintf(`{"input_tokens":%d}`, count)
|
||||
}
|
||||
|
||||
@@ -12,8 +12,9 @@ func init() {
|
||||
Codex,
|
||||
ConvertClaudeRequestToCodex,
|
||||
interfaces.TranslateResponse{
|
||||
Stream: ConvertCodexResponseToClaude,
|
||||
NonStream: ConvertCodexResponseToClaudeNonStream,
|
||||
Stream: ConvertCodexResponseToClaude,
|
||||
NonStream: ConvertCodexResponseToClaudeNonStream,
|
||||
TokenCount: ClaudeTokenCount,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user