feat: add client availability tracking and error handling improvements

- Introduced `IsAvailable` and `SetUnavailable` methods to clients for availability tracking.
- Integrated availability checks in client selection logic to skip unavailable clients.
- Enhanced error handling by marking clients unavailable on specific error codes (e.g., 401, 402).
- Removed redundant quota verification logs in client reordering logic.
This commit is contained in:
Luis Pater
2025-09-19 01:53:38 +08:00
parent 9ec8478b41
commit df66046b14
15 changed files with 183 additions and 21 deletions

View File

@@ -41,6 +41,9 @@ type ClientBase struct {
// modelRegistry is the global model registry for tracking model availability.
modelRegistry *registry.ModelRegistry
// unavailable tracks whether the client is unavailable
isAvailable bool
}
// GetRequestMutex returns the mutex used to synchronize requests for this client.