feat: introduce custom provider example and remove redundant debug logs

- Added `examples/custom-provider/main.go` showcasing custom executor and translator integration using the SDK. - Removed redundant debug logs from translator modules to enhance code cleanliness. - Updated SDK documentation with new usage and advanced examples. - Expanded the management API with new endpoints, including request logging and GPT-5 Codex features.
2026-02-02 04:20:50 +08:00 · 2025-09-22 03:37:53 +08:00
parent d5ad5fab87
commit f81898c906
54 changed files with 907 additions and 101 deletions
--- a/docs/sdk-advanced.md
+++ b/docs/sdk-advanced.md
@@ -0,0 +1,138 @@
+# SDK Advanced: Executors & Translators
+
+This guide explains how to extend the embedded proxy with custom providers and schemas using the SDK. You will:
+- Implement a provider executor that talks to your upstream API
+- Register request/response translators for schema conversion
+- Register models so they appear in `/v1/models`
+
+The examples use Go 1.24+ and the v6 module path.
+
+## Concepts
+
+- Provider executor: a runtime component implementing `auth.ProviderExecutor` that performs outbound calls for a given provider key (e.g., `gemini`, `claude`, `codex`). Executors can also implement `RequestPreparer` to inject credentials on raw HTTP requests.
+- Translator registry: schema conversion functions routed by `sdk/translator`. The built‑in handlers translate between OpenAI/Gemini/Claude/Codex formats; you can register new ones.
+- Model registry: publishes the list of available models per client/provider to power `/v1/models` and routing hints.
+
+## 1) Implement a Provider Executor
+
+Create a type that satisfies `auth.ProviderExecutor`.
+
+```go
+package myprov
+
+import (
+  "context"
+  "net/http"
+
+  coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
+  clipexec "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/executor"
+)
+
+type Executor struct{}
+
+func (Executor) Identifier() string { return "myprov" }
+
+// Optional: mutate outbound HTTP requests with credentials
+func (Executor) PrepareRequest(req *http.Request, a *coreauth.Auth) error {
+  // Example: req.Header.Set("Authorization", "Bearer "+a.APIKey)
+  return nil
+}
+
+func (Executor) Execute(ctx context.Context, a *coreauth.Auth, req clipexec.Request, opts clipexec.Options) (clipexec.Response, error) {
+  // Build HTTP request based on req.Payload (already translated into provider format)
+  // Use per‑auth transport if provided: transport := a.RoundTripper // via RoundTripperProvider
+  // Perform call and return provider JSON payload
+  return clipexec.Response{Payload: []byte(`{"ok":true}`)}, nil
+}
+
+func (Executor) ExecuteStream(ctx context.Context, a *coreauth.Auth, req clipexec.Request, opts clipexec.Options) (<-chan clipexec.StreamChunk, error) {
+  ch := make(chan clipexec.StreamChunk, 1)
+  go func() { defer close(ch); ch <- clipexec.StreamChunk{Payload: []byte("data: {\"done\":true}\n\n")} }()
+  return ch, nil
+}
+
+func (Executor) Refresh(ctx context.Context, a *coreauth.Auth) (*coreauth.Auth, error) {
+  // Optionally refresh tokens and return updated auth
+  return a, nil
+}
+```
+
+Register the executor with the core manager before starting the service:
+
+```go
+core := coreauth.NewManager(coreauth.NewFileStore(cfg.AuthDir), nil, nil)
+core.RegisterExecutor(myprov.Executor{})
+svc, _ := cliproxy.NewBuilder().WithConfig(cfg).WithConfigPath(cfgPath).WithCoreAuthManager(core).Build()
+```
+
+If your auth entries use provider `"myprov"`, the manager routes requests to your executor.
+
+## 2) Register Translators
+
+The handlers accept OpenAI/Gemini/Claude/Codex inputs. To support a new provider format, register translation functions in `sdk/translator`’s default registry.
+
+Direction matters:
+- Request: register from inbound schema to provider schema
+- Response: register from provider schema back to inbound schema
+
+Example: Convert OpenAI Chat → MyProv Chat and back.
+
+```go
+package myprov
+
+import (
+  "context"
+  sdktr "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
+)
+
+const (
+  FOpenAI = sdktr.Format("openai.chat")
+  FMyProv = sdktr.Format("myprov.chat")
+)
+
+func init() {
+  sdktr.Register(FOpenAI, FMyProv,
+    // Request transform (model, rawJSON, stream)
+    func(model string, raw []byte, stream bool) []byte { return convertOpenAIToMyProv(model, raw, stream) },
+    // Response transform (stream & non‑stream)
+    sdktr.ResponseTransform{
+      Stream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) []string {
+        return convertStreamMyProvToOpenAI(model, originalReq, translatedReq, raw)
+      },
+      NonStream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) string {
+        return convertMyProvToOpenAI(model, originalReq, translatedReq, raw)
+      },
+    },
+  )
+}
+```
+
+When the OpenAI handler receives a request that should route to `myprov`, the pipeline uses the registered transforms automatically.
+
+## 3) Register Models
+
+Expose models under `/v1/models` by registering them in the global model registry using the auth ID (client ID) and provider name.
+
+```go
+models := []*cliproxy.ModelInfo{
+  { ID: "myprov-pro-1", Object: "model", Type: "myprov", DisplayName: "MyProv Pro 1" },
+}
+cliproxy.GlobalModelRegistry().RegisterClient(authID, "myprov", models)
+```
+
+The embedded server calls this automatically for built‑in providers; for custom providers, register during startup (e.g., after loading auths) or upon auth registration hooks.
+
+## Credentials & Transports
+
+- Use `Manager.SetRoundTripperProvider` to inject per‑auth `*http.Transport` (e.g., proxy):
+  ```go
+  core.SetRoundTripperProvider(myProvider) // returns transport per auth
+  ```
+- For raw HTTP flows, implement `PrepareRequest` and/or call `Manager.InjectCredentials(req, authID)` to set headers.
+
+## Testing Tips
+
+- Enable request logging: Management API GET/PUT `/v0/management/request-log`
+- Toggle debug logs: Management API GET/PUT `/v0/management/debug`
+- Hot reload changes in `config.yaml` and `auths/` are picked up automatically by the watcher
+
--- a/docs/sdk-advanced_CN.md
+++ b/docs/sdk-advanced_CN.md
@@ -0,0 +1,131 @@
+# SDK 高级指南：执行器与翻译器
+
+本文介绍如何使用 SDK 扩展内嵌代理：
+- 实现自定义 Provider 执行器以调用你的上游 API
+- 注册请求/响应翻译器进行协议转换
+- 注册模型以出现在 `/v1/models`
+
+示例基于 Go 1.24+ 与 v6 模块路径。
+
+## 概念
+
+- Provider 执行器：实现 `auth.ProviderExecutor` 的运行时组件，负责某个 provider key（如 `gemini`、`claude`、`codex`）的真正出站调用。若实现 `RequestPreparer` 接口，可在原始 HTTP 请求上注入凭据。
+- 翻译器注册表：由 `sdk/translator` 驱动的协议转换函数。内置了 OpenAI/Gemini/Claude/Codex 的互转；你也可以注册新的格式转换。
+- 模型注册表：对外发布可用模型列表，供 `/v1/models` 与路由参考。
+
+## 1) 实现 Provider 执行器
+
+创建类型满足 `auth.ProviderExecutor` 接口。
+
+```go
+package myprov
+
+import (
+    "context"
+    "net/http"
+
+    coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
+    clipexec "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/executor"
+)
+
+type Executor struct{}
+
+func (Executor) Identifier() string { return "myprov" }
+
+// 可选：在原始 HTTP 请求上注入凭据
+func (Executor) PrepareRequest(req *http.Request, a *coreauth.Auth) error {
+    // 例如：req.Header.Set("Authorization", "Bearer "+a.Attributes["api_key"]) 
+    return nil
+}
+
+func (Executor) Execute(ctx context.Context, a *coreauth.Auth, req clipexec.Request, opts clipexec.Options) (clipexec.Response, error) {
+    // 基于 req.Payload 构造上游请求，返回上游 JSON 负载
+    return clipexec.Response{Payload: []byte(`{"ok":true}`)}, nil
+}
+
+func (Executor) ExecuteStream(ctx context.Context, a *coreauth.Auth, req clipexec.Request, opts clipexec.Options) (<-chan clipexec.StreamChunk, error) {
+    ch := make(chan clipexec.StreamChunk, 1)
+    go func() { defer close(ch); ch <- clipexec.StreamChunk{Payload: []byte("data: {\\"done\\":true}\\n\\n")} }()
+    return ch, nil
+}
+
+func (Executor) Refresh(ctx context.Context, a *coreauth.Auth) (*coreauth.Auth, error) { return a, nil }
+```
+
+在启动服务前将执行器注册到核心管理器：
+
+```go
+core := coreauth.NewManager(coreauth.NewFileStore(cfg.AuthDir), nil, nil)
+core.RegisterExecutor(myprov.Executor{})
+svc, _ := cliproxy.NewBuilder().WithConfig(cfg).WithConfigPath(cfgPath).WithCoreAuthManager(core).Build()
+```
+
+当凭据的 `Provider` 为 `"myprov"` 时，管理器会将请求路由到你的执行器。
+
+## 2) 注册翻译器
+
+内置处理器接受 OpenAI/Gemini/Claude/Codex 的入站格式。要支持新的 provider 协议，需要在 `sdk/translator` 的默认注册表中注册转换函数。
+
+方向很重要：
+- 请求：从“入站格式”转换为“provider 格式”
+- 响应：从“provider 格式”转换回“入站格式”
+
+示例：OpenAI Chat → MyProv Chat 及其反向。
+
+```go
+package myprov
+
+import (
+  "context"
+  sdktr "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
+)
+
+const (
+  FOpenAI = sdktr.Format("openai.chat")
+  FMyProv = sdktr.Format("myprov.chat")
+)
+
+func init() {
+  sdktr.Register(FOpenAI, FMyProv,
+    func(model string, raw []byte, stream bool) []byte { return convertOpenAIToMyProv(model, raw, stream) },
+    sdktr.ResponseTransform{
+      Stream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) []string {
+        return convertStreamMyProvToOpenAI(model, originalReq, translatedReq, raw)
+      },
+      NonStream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) string {
+        return convertMyProvToOpenAI(model, originalReq, translatedReq, raw)
+      },
+    },
+  )
+}
+```
+
+当 OpenAI 处理器接到需要路由到 `myprov` 的请求时，流水线会自动应用已注册的转换。
+
+## 3) 注册模型
+
+通过全局模型注册表将模型暴露到 `/v1/models`：
+
+```go
+models := []*cliproxy.ModelInfo{
+  { ID: "myprov-pro-1", Object: "model", Type: "myprov", DisplayName: "MyProv Pro 1" },
+}
+cliproxy.GlobalModelRegistry().RegisterClient(authID, "myprov", models)
+```
+
+内置 Provider 会自动注册；自定义 Provider 建议在启动时（例如加载到 Auth 后）或在 Auth 注册钩子中调用。
+
+## 凭据与传输
+
+- 使用 `Manager.SetRoundTripperProvider` 注入按账户的 `*http.Transport`（例如代理）：
+  ```go
+  core.SetRoundTripperProvider(myProvider) // 按账户返回 transport
+  ```
+- 对于原始 HTTP 请求，若实现了 `PrepareRequest`，或通过 `Manager.InjectCredentials(req, authID)` 进行头部注入。
+
+## 测试建议
+
+- 启用请求日志：管理 API GET/PUT `/v0/management/request-log`
+- 切换调试日志：管理 API GET/PUT `/v0/management/debug`
+- 热更新：`config.yaml` 与 `auths/` 变化会自动被侦测并应用
+
--- a/docs/sdk-usage.md
+++ b/docs/sdk-usage.md
@@ -0,0 +1,163 @@
+# CLI Proxy SDK Guide
+
+The `sdk/cliproxy` module exposes the proxy as a reusable Go library so external programs can embed the routing, authentication, hot‑reload, and translation layers without depending on the CLI binary.
+
+## Install & Import
+
+```bash
+go get github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy
+```
+
+```go
+import (
+    "context"
+    "errors"
+    "time"
+
+    "github.com/router-for-me/CLIProxyAPI/v6/internal/config"
+    "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy"
+)
+```
+
+Note the `/v6` module path.
+
+## Minimal Embed
+
+```go
+cfg, err := config.LoadConfig("config.yaml")
+if err != nil { panic(err) }
+
+svc, err := cliproxy.NewBuilder().
+    WithConfig(cfg).
+    WithConfigPath("config.yaml"). // absolute or working-dir relative
+    Build()
+if err != nil { panic(err) }
+
+ctx, cancel := context.WithCancel(context.Background())
+defer cancel()
+
+if err := svc.Run(ctx); err != nil && !errors.Is(err, context.Canceled) {
+    panic(err)
+}
+```
+
+The service manages config/auth watching, background token refresh, and graceful shutdown. Cancel the context to stop it.
+
+## Server Options (middleware, routes, logs)
+
+The server accepts options via `WithServerOptions`:
+
+```go
+svc, _ := cliproxy.NewBuilder().
+  WithConfig(cfg).
+  WithConfigPath("config.yaml").
+  WithServerOptions(
+    // Add global middleware
+    cliproxy.WithMiddleware(func(c *gin.Context) { c.Header("X-Embed", "1"); c.Next() }),
+    // Tweak gin engine early (CORS, trusted proxies, etc.)
+    cliproxy.WithEngineConfigurator(func(e *gin.Engine) { e.ForwardedByClientIP = true }),
+    // Add your own routes after defaults
+    cliproxy.WithRouterConfigurator(func(e *gin.Engine, _ *handlers.BaseAPIHandler, _ *config.Config) {
+      e.GET("/healthz", func(c *gin.Context) { c.String(200, "ok") })
+    }),
+    // Override request log writer/dir
+    cliproxy.WithRequestLoggerFactory(func(cfg *config.Config, cfgPath string) logging.RequestLogger {
+      return logging.NewFileRequestLogger(true, "logs", filepath.Dir(cfgPath))
+    }),
+  ).
+  Build()
+```
+
+These options mirror the internals used by the CLI server.
+
+## Management API (when embedded)
+
+- Management endpoints are mounted only when `remote-management.secret-key` is set in `config.yaml`.
+- Remote access additionally requires `remote-management.allow-remote: true`.
+- See MANAGEMENT_API.md for endpoints. Your embedded server exposes them under `/v0/management` on the configured port.
+
+## Using the Core Auth Manager
+
+The service uses a core `auth.Manager` for selection, execution, and auto‑refresh. When embedding, you can provide your own manager to customize transports or hooks:
+
+```go
+core := coreauth.NewManager(coreauth.NewFileStore(cfg.AuthDir), nil, nil)
+core.SetRoundTripperProvider(myRTProvider) // per‑auth *http.Transport
+
+svc, _ := cliproxy.NewBuilder().
+    WithConfig(cfg).
+    WithConfigPath("config.yaml").
+    WithCoreAuthManager(core).
+    Build()
+```
+
+Implement a custom per‑auth transport:
+
+```go
+type myRTProvider struct{}
+func (myRTProvider) RoundTripperFor(a *coreauth.Auth) http.RoundTripper {
+    if a == nil || a.ProxyURL == "" { return nil }
+    u, _ := url.Parse(a.ProxyURL)
+    return &http.Transport{ Proxy: http.ProxyURL(u) }
+}
+```
+
+Programmatic execution is available on the manager:
+
+```go
+// Non‑streaming
+resp, err := core.Execute(ctx, []string{"gemini"}, req, opts)
+
+// Streaming
+chunks, err := core.ExecuteStream(ctx, []string{"gemini"}, req, opts)
+for ch := range chunks { /* ... */ }
+```
+
+Note: Built‑in provider executors are wired automatically when you run the `Service`. If you want to use `Manager` stand‑alone without the HTTP server, you must register your own executors that implement `auth.ProviderExecutor`.
+
+## Custom Client Sources
+
+Replace the default loaders if your creds live outside the local filesystem:
+
+```go
+type memoryTokenProvider struct{}
+func (p *memoryTokenProvider) Load(ctx context.Context, cfg *config.Config) (*cliproxy.TokenClientResult, error) {
+    // Populate from memory/remote store and return counts
+    return &cliproxy.TokenClientResult{}, nil
+}
+
+svc, _ := cliproxy.NewBuilder().
+  WithConfig(cfg).
+  WithConfigPath("config.yaml").
+  WithTokenClientProvider(&memoryTokenProvider{}).
+  WithAPIKeyClientProvider(cliproxy.NewAPIKeyClientProvider()).
+  Build()
+```
+
+## Hooks
+
+Observe lifecycle without patching internals:
+
+```go
+hooks := cliproxy.Hooks{
+  OnBeforeStart: func(cfg *config.Config) { log.Infof("starting on :%d", cfg.Port) },
+  OnAfterStart:  func(s *cliproxy.Service) { log.Info("ready") },
+}
+svc, _ := cliproxy.NewBuilder().WithConfig(cfg).WithConfigPath("config.yaml").WithHooks(hooks).Build()
+```
+
+## Shutdown
+
+`Run` defers `Shutdown`, so cancelling the parent context is enough. To stop manually:
+
+```go
+ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+defer cancel()
+_ = svc.Shutdown(ctx)
+```
+
+## Notes
+
+- Hot reload: changes to `config.yaml` and `auths/` are picked up automatically.
+- Request logging can be toggled at runtime via the Management API.
+- Gemini Web features (`gemini-web.*`) are honored in the embedded server.
--- a/docs/sdk-usage_CN.md
+++ b/docs/sdk-usage_CN.md
@@ -0,0 +1,164 @@
+# CLI Proxy SDK 使用指南
+
+`sdk/cliproxy` 模块将代理能力以 Go 库的形式对外暴露，方便在其它服务中内嵌路由、鉴权、热更新与翻译层，而无需依赖可执行的 CLI 程序。
+
+## 安装与导入
+
+```bash
+go get github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy
+```
+
+```go
+import (
+    "context"
+    "errors"
+    "time"
+
+    "github.com/router-for-me/CLIProxyAPI/v6/internal/config"
+    "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy"
+)
+```
+
+注意模块路径包含 `/v6`。
+
+## 最小可用示例
+
+```go
+cfg, err := config.LoadConfig("config.yaml")
+if err != nil { panic(err) }
+
+svc, err := cliproxy.NewBuilder().
+    WithConfig(cfg).
+    WithConfigPath("config.yaml"). // 绝对路径或工作目录相对路径
+    Build()
+if err != nil { panic(err) }
+
+ctx, cancel := context.WithCancel(context.Background())
+defer cancel()
+
+if err := svc.Run(ctx); err != nil && !errors.Is(err, context.Canceled) {
+    panic(err)
+}
+```
+
+服务内部会管理配置与认证文件的监听、后台令牌刷新与优雅关闭。取消上下文即可停止服务。
+
+## 服务器可选项（中间件、路由、日志）
+
+通过 `WithServerOptions` 自定义：
+
+```go
+svc, _ := cliproxy.NewBuilder().
+  WithConfig(cfg).
+  WithConfigPath("config.yaml").
+  WithServerOptions(
+    // 追加全局中间件
+    cliproxy.WithMiddleware(func(c *gin.Context) { c.Header("X-Embed", "1"); c.Next() }),
+    // 提前调整 gin 引擎（如 CORS、trusted proxies）
+    cliproxy.WithEngineConfigurator(func(e *gin.Engine) { e.ForwardedByClientIP = true }),
+    // 在默认路由之后追加自定义路由
+    cliproxy.WithRouterConfigurator(func(e *gin.Engine, _ *handlers.BaseAPIHandler, _ *config.Config) {
+      e.GET("/healthz", func(c *gin.Context) { c.String(200, "ok") })
+    }),
+    // 覆盖请求日志的创建（启用/目录）
+    cliproxy.WithRequestLoggerFactory(func(cfg *config.Config, cfgPath string) logging.RequestLogger {
+      return logging.NewFileRequestLogger(true, "logs", filepath.Dir(cfgPath))
+    }),
+  ).
+  Build()
+```
+
+这些选项与 CLI 服务器内部用法保持一致。
+
+## 管理 API（内嵌时）
+
+- 仅当 `config.yaml` 中设置了 `remote-management.secret-key` 时才会挂载管理端点。
+- 远程访问还需要 `remote-management.allow-remote: true`。
+- 具体端点见 MANAGEMENT_API_CN.md。内嵌服务器会在配置端口下暴露 `/v0/management`。
+
+## 使用核心鉴权管理器
+
+服务内部使用核心 `auth.Manager` 负责选择、执行、自动刷新。内嵌时可自定义其传输或钩子：
+
+```go
+core := coreauth.NewManager(coreauth.NewFileStore(cfg.AuthDir), nil, nil)
+core.SetRoundTripperProvider(myRTProvider) // 按账户返回 *http.Transport
+
+svc, _ := cliproxy.NewBuilder().
+    WithConfig(cfg).
+    WithConfigPath("config.yaml").
+    WithCoreAuthManager(core).
+    Build()
+```
+
+实现每个账户的自定义传输：
+
+```go
+type myRTProvider struct{}
+func (myRTProvider) RoundTripperFor(a *coreauth.Auth) http.RoundTripper {
+    if a == nil || a.ProxyURL == "" { return nil }
+    u, _ := url.Parse(a.ProxyURL)
+    return &http.Transport{ Proxy: http.ProxyURL(u) }
+}
+```
+
+管理器提供编程式执行接口：
+
+```go
+// 非流式
+resp, err := core.Execute(ctx, []string{"gemini"}, req, opts)
+
+// 流式
+chunks, err := core.ExecuteStream(ctx, []string{"gemini"}, req, opts)
+for ch := range chunks { /* ... */ }
+```
+
+说明：运行 `Service` 时会自动注册内置的提供商执行器；若仅单独使用 `Manager` 而不启动 HTTP 服务器，则需要自行实现并注册满足 `auth.ProviderExecutor` 的执行器。
+
+## 自定义凭据来源
+
+当凭据不在本地文件系统时，替换默认加载器：
+
+```go
+type memoryTokenProvider struct{}
+func (p *memoryTokenProvider) Load(ctx context.Context, cfg *config.Config) (*cliproxy.TokenClientResult, error) {
+    // 从内存/远端加载并返回数量统计
+    return &cliproxy.TokenClientResult{}, nil
+}
+
+svc, _ := cliproxy.NewBuilder().
+  WithConfig(cfg).
+  WithConfigPath("config.yaml").
+  WithTokenClientProvider(&memoryTokenProvider{}).
+  WithAPIKeyClientProvider(cliproxy.NewAPIKeyClientProvider()).
+  Build()
+```
+
+## 启动钩子
+
+无需修改内部代码即可观察生命周期：
+
+```go
+hooks := cliproxy.Hooks{
+  OnBeforeStart: func(cfg *config.Config) { log.Infof("starting on :%d", cfg.Port) },
+  OnAfterStart:  func(s *cliproxy.Service) { log.Info("ready") },
+}
+svc, _ := cliproxy.NewBuilder().WithConfig(cfg).WithConfigPath("config.yaml").WithHooks(hooks).Build()
+```
+
+## 关闭
+
+`Run` 内部会延迟调用 `Shutdown`，因此只需取消父上下文即可。若需手动停止：
+
+```go
+ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+defer cancel()
+_ = svc.Shutdown(ctx)
+```
+
+## 说明
+
+- 热更新：`config.yaml` 与 `auths/` 变化会被自动侦测并应用。
+- 请求日志可通过管理 API 在运行时开关。
+- `gemini-web.*` 相关配置在内嵌服务器中会被遵循。
+