Commit Graph

313 Commits

Author SHA1 Message Date
hkfires
e8e00d4cb8 refactor(watcher): Remove unnecessary log separator 2025-09-23 10:05:59 +08:00
Luis Pater
e68a6037e2 feat(auth): enable model suspension and resumption logic in AuthManager
- Added model suspension with reason tracking for 401 (unauthorized) and 402/403 (payment-related) errors.
- Implemented resumption logic upon model quota recovery or auth state changes.
- Enhanced registry to manage suspended clients, including counts and observability data.
- Updated availability computation to exclude suspended clients, ensuring accurate client model tracking.
2025-09-23 09:24:55 +08:00
Luis Pater
6046a8c95b feat(auth): enhance watcher with asynchronous dispatch and buffering
- Added async dispatch loop to `Watcher` for handling incremental `AuthUpdate` with in-memory buffering.
- Improved resilience against high-frequency auth changes by coalescing updates and reducing redundant processing.
- Updated `cliproxy` service to increase auth update queue capacity and optimize backlog consumption.
- Added detailed SDK integration documentation in English and Chinese (`sdk-watcher.md`, `sdk-watcher_CN.md`).
2025-09-23 04:33:48 +08:00
Luis Pater
792ec49e5b feat(auth): implement incremental auth updates with queue integration
- Added support for incremental auth updates using `AuthUpdate` and `AuthUpdateAction`.
- Integrated `SetAuthUpdateQueue` to propagate updates through a dedicated channel.
- Introduced new methods for handling auth add, modify, and delete actions.
- Updated service to ensure auth update queues are correctly initialized and consumed.
- Improved auth state synchronization across core and file-based clients with real-time updates.
- Refactored redundant auth handling logic for better efficiency and maintainability.
2025-09-23 04:16:22 +08:00
Luis Pater
e313d39be8 feat(gemini-executor): implement CountTokens method with request translation and API integration
- Added `CountTokens` for token counting requests in Gemini executor.
- Integrated request translation via `sdktranslator` and response handling.
- Improved error handling, logging, and API request configuration with headers.
2025-09-23 02:45:08 +08:00
Luis Pater
ac59023abb feat(executor): add CountTokens support across all executors
- Introduced `CountTokens` method to Codex, Claude, Gemini, Qwen, OpenAI-compatible, and other executors.
- Implemented `ExecuteCount` in `AuthManager` for token counting via provider round-robin.
- Updated handlers to leverage `ExecuteCountWithAuthManager` for streamlined token counting.
- Added fallback and error handling logic for token counting requests.
2025-09-23 02:27:51 +08:00
Luis Pater
d32fc0400e refactor(headers): centralize header logic using EnsureHeader utility
- Introduced `EnsureHeader` in `internal/misc/header_utils.go` to streamline header setting across executors.
- Updated Codex, Claude, and Gemini executors to utilize `EnsureHeader` for consistent header application.
- Incorporated Gin context headers (if available) into request header manipulation for better integration.
2025-09-23 02:01:57 +08:00
Luis Pater
7ea88358f0 refactor(executor): centralize header application logic for executors
- Replaced repetitive header setting logic with helper methods (`applyCodexHeaders`, `applyClaudeHeaders`, `applyQwenHeaders`) in Codex, Claude, and Qwen executors.
- Ensured consistent headers in HTTP requests across all executors.
- Introduced UUID and additional structured headers for better traceability (e.g., session IDs, metadata).
2025-09-23 01:20:10 +08:00
Luis Pater
c6b391304d chore(executor): add debug logging for API request errors
- Added detailed debug logs in all executors (Codex, Claude, Gemini, Qwen, OpenAI-compatible) to capture HTTP status and response body for failed API requests.
2025-09-22 23:37:53 +08:00
Luis Pater
2e836cee88 feat(auth): standardize last_refresh metadata handling across executors
- Added `last_refresh` timestamp to metadata for Codex, Claude, Qwen, and Gemini executors.
- Implemented `extractLastRefreshTimestamp` utility for parsing diverse timestamp formats in management handlers.
- Ensured consistent update and preservation of `last_refresh` in file-based auth handling.
2025-09-22 23:23:31 +08:00
Luis Pater
e41d127732 feat(openai-compat): enhance provider key handling and model resolution
- Introduced dynamic `providerKey` resolution for OpenAI-compatible providers, incorporating attributes like `provider_key` and `compat_name`.
- Implemented upstream model overrides via `resolveUpstreamModel` and `overrideModel` methods in the OpenAI executor.
- Updated registry logic to correctly store provider mappings and register clients using normalized keys.
- Ensured consistency in handling empty or default provider names across components.
2025-09-22 22:54:21 +08:00
Luis Pater
f1c4caf14a feat(logging): integrate logrus with custom Gin middleware for enhanced request logging and recovery
- Added `GinLogrusLogger` for structured request logging using Logrus.
- Implemented `GinLogrusRecovery` to handle panics and log stack traces.
- Configured log rotation using Lumberjack for efficient log management.
- Replaced Gin's default logger and recovery middleware with the custom implementations.
2025-09-22 22:17:12 +08:00
hkfires
22a69333a0 feat(gemini-web): Implement proactive PSIDTS cookie rotation 2025-09-22 21:54:52 +08:00
Luis Pater
ed87dda0a6 refactor(client): remove deprecated and unused methods for cleaner session handling and core auth management 2025-09-22 21:22:32 +08:00
Luis Pater
053134f66e refactor(auth): remove unused Refresh methods from authenticators
- Deleted `Refresh` implementations in Codex, Claude, Gemini, Qwen, and Gemini-web authenticators.
- Updated the `Authenticator` interface to exclude `Refresh` for cleaner design.
- Revised `Manager` and related components to handle refresh logic improvements.
- Simplified token refresh behavior and eliminated redundant code paths.
2025-09-22 21:11:53 +08:00
Luis Pater
837ae1b1b3 chore(logging): add debug logs for executor Refresh methods
- Introduced `logrus` for structured debugging across all executors.
- Added debug log messages in `Refresh` methods for better traceability.
- Updated `Manager` to log additional details during refresh checks.
2025-09-22 20:03:31 +08:00
Luis Pater
4008be19f4 feat(auth): introduce auth.providers for flexible authentication configuration
- Replaced legacy `api-keys` field with `auth.providers` in configuration, supporting multiple authentication providers including `config-api-key`.
- Added synchronization to maintain compatibility with legacy `api-keys`.
- Updated core components like request handling and middleware to use the new provider system.
- Enhanced management API endpoints for seamless integration with `auth.providers`.
2025-09-22 17:36:31 +08:00
hkfires
314125e7ec fix(logging): Prevent race conditions in FileStreamingLogWriter 2025-09-22 16:46:01 +08:00
hkfires
759bb88a90 fix(auth): Improve file-based auth handling and consistency 2025-09-22 13:03:47 +08:00
Luis Pater
0607e52767 feat: implement token refresh support for executors
- Added `Refresh` method implementations for Codex, Claude, Gemini, and Qwen executors.
- Introduced OAuth-based token handling for Gemini and Qwen with support for refresh tokens.
- Updated Codex and Claude to use new internal auth services.
- Enhanced metadata structure and consistency for token storage across all executors.
2025-09-22 09:27:03 +08:00
hkfires
d6bb143978 refactor(translator): Remove unused logrus imports 2025-09-22 08:01:37 +08:00
Luis Pater
f81898c906 feat: introduce custom provider example and remove redundant debug logs
- Added `examples/custom-provider/main.go` showcasing custom executor and translator integration using the SDK.
- Removed redundant debug logs from translator modules to enhance code cleanliness.
- Updated SDK documentation with new usage and advanced examples.
- Expanded the management API with new endpoints, including request logging and GPT-5 Codex features.
2025-09-22 03:37:53 +08:00
Luis Pater
d5ad5fab87 feat: enhance request logging and account handling in CLI proxy
- Added helper functions to log API request and response payloads in the Gin context.
- Improved `AccountInfo` to support cookie-based authentication in addition to API key and OAuth.
- Updated log messages for better clarity on account types used.
2025-09-22 03:09:15 +08:00
Luis Pater
d9ad65622a refactor: standardize constant naming and improve file-based auth handling
- Renamed constants from uppercase to CamelCase for consistency.
- Replaced redundant file-based auth handling logic with the new `util.CountAuthFiles` helper.
- Fixed various error-handling inconsistencies and enhanced robustness in file operations.
- Streamlined auth client reload logic in server and watcher components.
- Applied minor code readability improvements across multiple packages.
2025-09-22 02:56:45 +08:00
Luis Pater
4999fce7f4 v6 version first commit 2025-09-22 01:40:24 +08:00
Luis Pater
d42384cdb7 Merge branch 'dev' 2025-09-20 01:38:36 +08:00
Luis Pater
24f243a1bc feat: add support for Gemini 2.5 Flash image preview alias
- Introduced `gemini-2.5-flash-image-preview` alias in `GeminiWebAliasMap` for enhanced model handling.
- Added `gemini-2.5-flash-image-preview` as a new model variant with custom ID, name, display name, and description.
2025-09-20 01:37:42 +08:00
hkfires
aba719f5fe refactor(auth): Centralize auth file reading with snapshot preference
The logic for reading authentication files, which includes retries and a preference for cookie snapshot files, was previously implemented locally within the `watcher` package. This was done to handle potential file locks during writes.

This change moves this functionality into a shared `ReadAuthFileWithRetry` function in the `util` package to promote code reuse and consistency.

The `watcher` package is updated to use this new centralized function. Additionally, the initial token loading in the `run` command now also uses this logic, making it more resilient to file access issues and consistent with the watcher's behavior.
2025-09-20 00:14:26 +08:00
hkfires
1d7abc95b8 fix(gemini-web): ensure colon spacing in JSON output for compatibility 2025-09-19 23:32:52 +08:00
hkfires
395164e2d4 feat(log): Add separator when saving client credentials 2025-09-19 12:36:17 +08:00
hkfires
4bfafbe3aa refactor(watcher): Move API key client creation to watcher package 2025-09-19 11:46:17 +08:00
hkfires
2274d7488b refactor(auth): Centralize logging for saving credentials
The logic for logging the path where credentials are saved was duplicated across several client implementations.

This commit refactors this behavior by creating a new centralized function, `misc.LogSavingCredentials`, to handle this logging. The `SaveTokenToFile` method in each authentication token storage struct now calls this new function, ensuring consistent logging and reducing code duplication.

The redundant logging statements in the client-level `SaveTokenToFile` methods have been removed.
2025-09-19 11:46:17 +08:00
hkfires
39518ec633 refactor(client): Improve auth file handling and client lifecycle 2025-09-19 11:46:17 +08:00
hkfires
6bd37b2a2b fix(client): Prevent overwriting auth file on update 2025-09-19 11:46:16 +08:00
hkfires
f17ec7ffd8 fix(client): Prevent overwriting auth file on update 2025-09-19 11:46:16 +08:00
hkfires
d9f8129a32 fix(client): Add reason to unregistration to skip persistence 2025-09-19 11:46:16 +08:00
hkfires
8f0a345e2a refactor(watcher): Filter irrelevant file system events early 2025-09-19 11:46:16 +08:00
hkfires
56b2dabcca refactor(auth): Introduce generic cookie snapshot manager
This commit introduces a generic `cookies.Manager` to centralize the logic for handling cookie snapshots, which was previously duplicated across the Gemini and PaLM clients. This refactoring eliminates code duplication and improves maintainability.

The new `cookies.Manager[T]` in `internal/auth/cookies` orchestrates the lifecycle of cookie data between a temporary snapshot file and the main token file. It provides `Apply`, `Persist`, and `Flush` methods to manage this process.

Key changes:
- A generic `Manager` is created in `internal/auth/cookies`, usable for any token storage type.
- A `Hooks` struct allows for customizable behavior, such as custom merging strategies for different token types.
- Duplicated snapshot handling code has been removed from the `gemini-web` and `palm` persistence packages.
- The `GeminiWebClient` and `PaLMClient` have been updated to use the new `cookies.Manager`.
- The `auth_gemini` and `auth_palm` CLI commands now leverage the client's `Flush` method, simplifying the command logic.
- Cookie snapshot utility functions have been moved from `internal/util/files.go` to a new `internal/util/cookies.go` for better organization.
2025-09-19 11:46:09 +08:00
hkfires
7632204966 refactor(cookie): Extract cookie snapshot logic to util package
The logic for managing cookie persistence files was previously implemented directly within the `gemini-web` client's persistence layer. This approach was not reusable and led to duplicated helper functions.

This commit refactors the cookie persistence mechanism by:
- Renaming the concept from "sidecar" to "snapshot" for clarity.
- Extracting file I/O and path manipulation logic into a new, generic `internal/util/cookie_snapshot.go` file.
- Creating reusable utility functions: `WriteCookieSnapshot`, `TryReadCookieSnapshotInto`, and `RemoveCookieSnapshot`.
- Updating the `gemini-web` persistence code to use these new centralized utility functions.

This change improves code organization, reduces duplication, and makes the cookie snapshot functionality easier to maintain and potentially reuse across other clients.
2025-09-19 11:44:27 +08:00
Luis Pater
c0fbc1979e refactor: remove redundant Codex instruction validation logic
- Eliminated unnecessary calls to `misc.CodexInstructions` and corresponding checks in response processing.
- Streamlined instruction handling by directly updating "instructions" field from the original request payload.
2025-09-19 09:19:18 +08:00
Luis Pater
d00604dd28 Refine Codex instructions for GPT-5 use case
- Simplified `gpt_5_instructions.txt` by removing redundancy and aligning scope with AGENTS.md standard instructions.
- Enhanced clarity and relevance of directives per Codex context.
2025-09-19 09:09:22 +08:00
Luis Pater
869a3dfbb4 feat: implement model-specific Codex instructions for GPT-5
- Added `CodexInstructions(modelName string)` function to dynamically select instructions based on the model (e.g., GPT-5 Codex).
- Introduced `gpt_5_instructions.txt` and `gpt_5_codex_instructions.txt` for respective model configurations.
- Updated translators to pass `modelName` and use the new instruction logic.
2025-09-19 08:47:54 +08:00
Luis Pater
df66046b14 feat: add client availability tracking and error handling improvements
- Introduced `IsAvailable` and `SetUnavailable` methods to clients for availability tracking.
- Integrated availability checks in client selection logic to skip unavailable clients.
- Enhanced error handling by marking clients unavailable on specific error codes (e.g., 401, 402).
- Removed redundant quota verification logs in client reordering logic.
2025-09-19 01:53:38 +08:00
hkfires
bb6ec7ca81 fix(gemini-web): Correct inaccurate cookie refresh log message 2025-09-18 12:35:39 +08:00
hkfires
1b2e3dc7af feat(gemini): Implement pseudo-streaming and improve context reuse
This commit introduces two major enhancements to the Gemini Web client to improve user experience and conversation continuity.

First, it implements a pseudo-streaming mechanism for non-code mode. The Gemini Web API returns the full response at once in this mode, leading to a poor user experience with a long wait for output. This change splits the full response into smaller chunks and sends them with an 80ms delay, simulating a real-time streaming effect.

Second, the conversation context reuse logic is now more robust. A fallback mechanism has been added to reuse conversation metadata when a clear continuation context is detected (e.g., a user replies to an assistant's turn). This improves conversational flow. Metadata lookups have also been improved to check both the canonical model key and its alias for better compatibility.
2025-09-18 11:22:56 +08:00
hkfires
e4dd22b260 feat(gemini-web): squash all features and fixes for gemini-web 2025-09-17 20:24:23 +08:00
Luis Pater
172f282e9e feat: add graceful shutdown for streaming response handling
- Introduced `streamDone` channel to signal the completion of streaming goroutines.
- Updated `processStreamingChunks` to incorporate proper cleanup and defer close operations.
- Ensured `streamWriter` and associated resources are released when streaming completes.
2025-09-17 19:46:57 +08:00
Luis Pater
7f0c9b1942 Merge pull request #43 from kyle-the-dev/fix/gemini-schema-sanitization
fix: comprehensive JSON Schema sanitization for Claude to Gemini
2025-09-16 20:26:32 +08:00
Luis Pater
8c2063aeea Fix log key name from gpt-5-codex to force-gpt-5-codex for consistency 2025-09-16 20:03:48 +08:00
Kyle Ryan
a0c389a854 fix: comprehensive JSON Schema sanitization for Claude to Gemini
- Add SanitizeSchemaForGemini utility handling union types, allOf, exclusiveMinimum
- Fix both gemini-cli and gemini API translators
- Resolve "Proto field is not repeating, cannot start list" errors
- Maintain backward compatibility with fallback logic

This fixes Claude Code CLI compatibility issues when using tools with either
Gemini CLI credentials or direct Gemini API keys by properly sanitizing
JSON Schema fields that are incompatible with Gemini's Protocol Buffer validation.
2025-09-16 18:24:49 +08:00