chuan/CLIProxyAPI
1
0
Fork 0
mirror of https://github.com/router-for-me/CLIProxyAPI.git synced 2026-02-03 04:50:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add OpenAI compatibility support and improve resource cleanup

Browse Source 
- Introduced OpenAI compatibility configurations for external providers, enabling model alias routing via the OpenAI API format.
- Enhanced provider logic in `GetProviderName` to handle OpenAI aliases and added new helper functions for compatibility checks.
- Updated API handlers and client initialization to support OpenAI compatibility models.
- Improved resource cleanup across clients by closing response bodies and streams using deferred functions.
This commit is contained in:
Luis Pater
2025-08-26 03:21:19 +08:00
parent 9102ff031d
commit ed8873fbb0
18 changed files with 671 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files

109
README.md
View File

@@ -6,9 +6,9 @@ A proxy server that provides OpenAI/Gemini/Claude compatible API interfaces for
It now also supports OpenAI Codex (GPT models) and Claude Code via OAuth.
so you can use local or multi‑account CLI access with OpenAI‑compatible clients and SDKs.
So you can use local or multi-account CLI access with OpenAI-compatible clients and SDKs.
Now, We added the first Chinese provider: [Qwen Code](https://github.com/QwenLM/qwen-code).
The first Chinese provider has now been added: [Qwen Code](https://github.com/QwenLM/qwen-code).
## Features
@@ -19,12 +19,13 @@ Now, We added the first Chinese provider: [Qwen Code](https://github.com/QwenLM/
- Streaming and non-streaming responses
- Function calling/tools support
- Multimodal input support (text and images)
- Multiple accounts with round‑robin load balancing (Gemini, OpenAI, Claude and Qwen)
- Multiple accounts with round-robin load balancing (Gemini, OpenAI, Claude and Qwen)
- Simple CLI authentication flows (Gemini, OpenAI, Claude and Qwen)
- Generative Language API Key support
- Gemini CLI multi‑account load balancing
- Claude Code multi‑account load balancing
- Qwen Code multi‑account load balancing
- Gemini CLI multi-account load balancing
- Claude Code multi-account load balancing
- Qwen Code multi-account load balancing
- OpenAI-compatible upstream providers via config (e.g., OpenRouter)
## Installation
@@ -59,13 +60,13 @@ You can authenticate for Gemini, OpenAI, and/or Claude. All can coexist in the s
```bash
./cli-proxy-api --login
```
If you are an old gemini code user, you may need to specify a project ID:
If you are an existing Gemini Code user, you may need to specify a project ID:
```bash
./cli-proxy-api --login --project_id <your_project_id>
```
The local OAuth callback uses port `8085`.
Options: add `--no-browser` to print the login URL instead of opening a browser. The local OAuth callback uses port `1455`.
Options: add `--no-browser` to print the login URL instead of opening a browser. The local OAuth callback uses port `8085`.
- OpenAI (Codex/GPT via OAuth):
```bash
@@ -126,7 +127,7 @@ Request body example:
```
Notes:
- Use a `gemini-*` model for Gemini (e.g., `gemini-2.5-pro`), a `gpt-*` model for OpenAI (e.g., `gpt-5`), a `claude-*` model for Claude (e.g., `claude-3-5-sonnet-20241022`), or a `qwen-*` model for Qwen (e.g., `qwen3-coder-plus`). The proxy will route to the correct provider automatically.
- Use a `gemini-*` model for Gemini (e.g., "gemini-2.5-pro"), a `gpt-*` model for OpenAI (e.g., "gpt-5"), a `claude-*` model for Claude (e.g., "claude-3-5-sonnet-20241022"), or a `qwen-*` model for Qwen (e.g., "qwen3-coder-plus"). The proxy will route to the correct provider automatically.
#### Claude Messages (SSE-compatible)
@@ -226,7 +227,7 @@ console.log(await claudeResponse.json());
- claude-3-5-haiku-20241022
- qwen3-coder-plus
- qwen3-coder-flash
- Gemini models auto‑switch to preview variants when needed
- Gemini models auto-switch to preview variants when needed
## Configuration
@@ -238,21 +239,28 @@ The server uses a YAML configuration file (`config.yaml`) located in the project
### Configuration Options
| Parameter | Type | Default | Description |
|---------------------------------------|----------|--------------------|-----------------------------------------------------------------------------------------------|
| `port` | integer | 8317 | The port number on which the server will listen |
| `auth-dir` | string | "~/.cli-proxy-api" | Directory where authentication tokens are stored. Supports using `~` for home directory |
| `proxy-url` | string | "" | Proxy url, support socks5/http/https protocol, example: socks5://user:pass@192.168.1.1:1080/ |
| `request-retry` | integer | 0 | Request retry times, if the response http code is 403, 408, 500, 502, 503, 504, it will retry |
| `quota-exceeded` | object | {} | Configuration for handling quota exceeded |
| `quota-exceeded.switch-project` | boolean | true | Whether to automatically switch to another project when a quota is exceeded |
| `quota-exceeded.switch-preview-model` | boolean | true | Whether to automatically switch to a preview model when a quota is exceeded |
| `debug` | boolean | false | Enable debug mode for verbose logging |
| `api-keys` | string[] | [] | List of API keys that can be used to authenticate requests |
| `generative-language-api-key` | string[] | [] | List of Generative Language API keys |
| `claude-api-key` | object | {} | List of Claude API keys |
| `claude-api-key.api-key` | string | "" | Claude API key |
| `claude-api-key.base-url` | string | "" | Custom Claude API endpoint, if you use the third party API endpoint |
| Parameter | Type | Default | Description |
|-----------------------------------------|----------|--------------------|---------------------------------------------------------------------------------------------------------|
| `port` | integer | 8317 | The port number on which the server will listen. |
| `auth-dir` | string | "~/.cli-proxy-api" | Directory where authentication tokens are stored. Supports using `~` for the home directory. |
| `proxy-url` | string | "" | Proxy URL. Supports socks5/http/https protocols. Example: socks5://user:pass@192.168.1.1:1080/ |
| `request-retry` | integer | 0 | Number of times to retry a request. Retries will occur if the HTTP response code is 403, 408, 500, 502, 503, or 504. |
| `quota-exceeded` | object | {} | Configuration for handling quota exceeded. |
| `quota-exceeded.switch-project` | boolean | true | Whether to automatically switch to another project when a quota is exceeded. |
| `quota-exceeded.switch-preview-model` | boolean | true | Whether to automatically switch to a preview model when a quota is exceeded. |
| `debug` | boolean | false | Enable debug mode for verbose logging. |
| `api-keys` | string[] | [] | List of API keys that can be used to authenticate requests. |
| `generative-language-api-key` | string[] | [] | List of Generative Language API keys. |
| `claude-api-key` | object | {} | List of Claude API keys. |
| `claude-api-key.api-key` | string | "" | Claude API key. |
| `claude-api-key.base-url` | string | "" | Custom Claude API endpoint, if you use a third-party API endpoint. |
| `openai-compatibility` | object[] | [] | Upstream OpenAI-compatible providers configuration (name, base-url, api-keys, models). |
| `openai-compatibility.*.name` | string | "" | The name of the provider. It will be used in the user agent and other places. |
| `openai-compatibility.*.base-url` | string | "" | The base URL of the provider. |
| `openai-compatibility.*.api-keys` | string[] | [] | The API keys for the provider. Add multiple keys if needed. Omit if unauthenticated access is allowed. |
| `openai-compatibility.*.models` | object[] | [] | The actual model name. |
| `openai-compatibility.*.models.*.name` | string | "" | The models supported by the provider. |
| `openai-compatibility.*.models.*.alias` | string | "" | The alias used in the API. |
### Example Configuration File
@@ -266,10 +274,10 @@ auth-dir: "~/.cli-proxy-api"
# Enable debug logging
debug: false
# Proxy url, support socks5/http/https protocol, example: socks5://user:pass@192.168.1.1:1080/
# Proxy URL. Supports socks5/http/https protocols. Example: socks5://user:pass@192.168.1.1:1080/
proxy-url: ""
# Request retry times, if the response http code is 403, 408, 500, 502, 503, 504, it will retry
# Number of times to retry a request. Retries will occur if the HTTP response code is 403, 408, 500, 502, 503, or 504.
request-retry: 3
# Quota exceeded behavior
@@ -294,8 +302,51 @@ claude-api-key:
- api-key: "sk-atSM..." # use the official claude API key, no need to set the base url
- api-key: "sk-atSM..."
base-url: "https://www.example.com" # use the custom claude API endpoint
# OpenAI compatibility providers
openai-compatibility:
- name: "openrouter" # The name of the provider; it will be used in the user agent and other places.
base-url: "https://openrouter.ai/api/v1" # The base URL of the provider.
api-keys: # The API keys for the provider. Add multiple keys if needed. Omit if unauthenticated access is allowed.
- "sk-or-v1-...b780"
- "sk-or-v1-...b781"
models: # The models supported by the provider.
- name: "moonshotai/kimi-k2:free" # The actual model name.
alias: "kimi-k2" # The alias used in the API.
```
### OpenAI Compatibility Providers
Configure upstream OpenAI-compatible providers (e.g., OpenRouter) via `openai-compatibility`.
- name: provider identifier used internally
- base-url: provider base URL
- api-keys: optional list of API keys (omit if provider allows unauthenticated requests)
- models: list of mappings from upstream model `name` to local `alias`
Example:
```yaml
openai-compatibility:
- name: "openrouter"
base-url: "https://openrouter.ai/api/v1"
api-keys:
- "sk-or-v1-...b780"
- "sk-or-v1-...b781"
models:
- name: "moonshotai/kimi-k2:free"
alias: "kimi-k2"
```
Usage:
Call OpenAI's endpoint `/v1/chat/completions` with `model` set to the alias (e.g., `kimi-k2`). The proxy routes to the configured provider/model automatically.
Also, you may call Claude's endpoint `/v1/messages`, Gemini's `/v1beta/models/model-name:streamGenerateContent` or `/v1beta/models/model-name:generateContent`.
And you can always use Gemini CLI with `CODE_ASSIST_ENDPOINT` set to `http://127.0.0.1:8317` for these OpenAI-compatible provider's models.
### Authentication Directory
The `auth-dir` parameter specifies where authentication tokens are stored. When you run the login command, the application will create JSON files in this directory containing the authentication tokens for your Google accounts. Multiple accounts can be used for load balancing.
@@ -327,8 +378,8 @@ export CODE_ASSIST_ENDPOINT="http://127.0.0.1:8317"
The server will relay the `loadCodeAssist`, `onboardUser`, and `countTokens` requests. And automatically load balance the text generation requests between the multiple accounts.
> [!NOTE]
> This feature only allows local access because I couldn't find a way to authenticate the requests.
> I hardcoded `127.0.0.1` into the load balancing.
> This feature only allows local access because there is currently no way to authenticate the requests.
> 127.0.0.1 is hardcoded for load balancing.
## Claude Code with multiple account load balancing