62 Commits

• Fix image limits test to use realistic payload sizes

  Previous test used compressed 8k images (0.01MB) which was meaningless.
  Now tests with actual large noise images that don't compress.
  
  Realistic payload limits discovered:
  - Anthropic: 6 x 3MB = ~18MB total (not 32MB as documented)
  - OpenAI: 2 x 15MB = ~30MB total
  - Gemini: 10 x 20MB = ~200MB total (very permissive)
  - Mistral: 4 x 10MB = ~40MB total
  - xAI: 1 x 20MB (strict request size limit)
  - Groq: 5 x 5760px images (5 image + pixel limit)
  - zAI: 2 x 15MB = ~30MB (50MB request limit)
  - OpenRouter: 2 x 5MB = ~10MB total
  
  Also fixed GEMINI_API_KEY env var (was GOOGLE_API_KEY).
  
  Related to #120

  Mario Zechner · 2025-12-16 23:48:59 +01:00
  e1ce9c1f49
• Update image limits test with comprehensive 8k stress test results

  Tested max 8kx8k images per provider:
  - Anthropic: 100 (explicit limit, fails at 101)
  - OpenAI: 100-200 (100 works, 200 times out)
  - Mistral: 8 (explicit limit, fails at 9)
  - xAI: 100-150 (100 works, 150 times out)
  - Groq: 0 (8k exceeds 33M pixel limit)
  - zAI: 400 (context window limited at 500)
  - OpenRouter: 40 (context window limited at 50)
  - Gemini: untested (no API key in test env)
  
  Key finding: Anthropic's 'many images' rule does NOT cause API errors.
  100 x 8kx8k images work fine. Anthropic likely auto-resizes internally.
  
  Related to #120

  Mario Zechner · 2025-12-16 23:01:46 +01:00
  043a8416b0
• Add comprehensive image limits test suite for all vision-capable providers

  Tests max image count, size, dimensions, and 8k stress test for:
  - Anthropic, OpenAI, Gemini, Mistral, OpenRouter, xAI, Groq, zAI
  
  Key finding: Anthropic's 'many images' rule (>20 images = 2000px max)
  does NOT cause API errors. 100 x 8k images work fine. Anthropic likely
  auto-resizes internally.
  
  Related to #120

  Mario Zechner · 2025-12-16 22:21:48 +01:00
  f1df52ccfd
• ai: add image limits test suite

  Tests provider-specific image limitations across all supported providers:
  - Maximum number of images in context
  - Maximum image size (bytes)
  - Maximum image dimensions
  
  Discovered limits (Dec 2025):
  - Anthropic: 100 images, 5MB per image, 8000px max dimension
  - OpenAI: 500 images, >=25MB per image
  - Gemini: ~2500 images, >=40MB per image
  - Mistral: 8 images, ~15MB per image
  - OpenRouter: ~40 images (context limited), ~15MB per image

  Mario Zechner · 2025-12-16 20:04:34 +01:00
  3c9c47d3bb
• Add X-Initiator header for GitHub Copilot (#200)
  Ahmed Kamal · 2025-12-16 14:05:22 +01:00
  c2dea0ce8b
• Read GitHub Copilot token from oauth.json in test
  Mario Zechner · 2025-12-15 19:16:08 +01:00
  f8550a536e
• Add GitHub Copilot test to packages/ai
  Mario Zechner · 2025-12-15 19:12:43 +01:00
  5a59b8d18d
• Fix branch selector for single message and --no-session mode

  - Allow branch selector to open with single user message (changed <= 1 to === 0 check)
  - Support in-memory branching for --no-session mode (no files created)
  - Add isEnabled() getter to SessionManager
  - Update sessionFile getter to return null when sessions disabled
  - Update SessionSwitchEvent types to allow null session files
  - Add branching tests for single message and --no-session scenarios
  
  fixes #163

  Mario Zechner · 2025-12-10 22:41:32 +01:00
  3d35e7c469
• Fix Mistral 400 errors after aborted assistant messages

  - Skip empty assistant messages (no content, no tool calls) to avoid
    Mistral's 'Assistant message must have either content or tool_calls'
    error
  - Remove synthetic assistant bridge message after tool results (Mistral
    no longer requires this as of Dec 2024)
  - Add test for empty assistant message handling
  
  Follow-up to #165

  Mario Zechner · 2025-12-10 21:13:33 +01:00
  76312ea7e8
• Add Mistral as AI provider

  - Add Mistral to KnownProvider type and model generation
  - Implement Mistral-specific compat handling in openai-completions:
    - requiresToolResultName: tool results need name field
    - requiresAssistantAfterToolResult: synthetic assistant message between tool/user
    - requiresThinkingAsText: thinking blocks as <thinking> text
    - requiresMistralToolIds: tool IDs must be exactly 9 alphanumeric chars
  - Add MISTRAL_API_KEY environment variable support
  - Add Mistral tests across all test files
  - Update documentation (README, CHANGELOG) for both ai and coding-agent packages
  - Remove client IDs from gemini.md, reference upstream source instead
  
  Closes #165

  Mario Zechner · 2025-12-10 20:36:19 +01:00
  99b4b1aca0
• Simplify compaction: remove proactive abort, use Agent.continue() for retry

  - Add agentLoopContinue() to pi-ai for resuming from existing context
  - Add Agent.continue() method and transport.continue() interface
  - Simplify AgentSession compaction to two cases: overflow (auto-retry) and threshold (no retry)
  - Remove proactive mid-turn compaction abort
  - Merge turn prefix summary into main summary
  - Add isCompacting property to AgentSession and RPC state
  - Block input during compaction in interactive mode
  - Show compaction count on session resume
  - Rename RPC.md to rpc.md for consistency
  
  Related to #128

  Mario Zechner · 2025-12-09 21:43:49 +01:00
  5a9d844f9a
• Fix tsgo type issues: update tsgo, fix ReasoningEffort import, remove broken enum-test
  Mario Zechner · 2025-12-08 22:59:13 +01:00
  238c5d34e4
• Add xhigh thinking level for OpenAI codex-max models

  - Add 'xhigh' to ThinkingLevel type in ai and agent packages
  - Map xhigh to reasoning_effort: 'max' for OpenAI providers
  - Add thinkingXhigh color token to theme schema and built-in themes
  - Show xhigh option only when using codex-max models
  - Update CHANGELOG for both ai and coding-agent packages
  
  closes #143

  Mario Zechner · 2025-12-08 21:12:54 +01:00
  00370cab39
• Implement tool result truncation with actionable notices (#134)

  - read: actionable notices with offset for continuation
    - First line > 30KB: return empty + bash command suggestion
    - Hit limit: '[Showing lines X-Y of Z. Use offset=N to continue]'
  
  - bash: tail truncation with temp file
    - Notice includes line range + temp file path
    - Edge case: last line > 30KB shows partial
  
  - grep: pre-truncate match lines to 500 chars
    - '[... truncated]' suffix on long lines
    - Notice for match limit and line truncation
  
  - find/ls: result/entry limit notices
    - '[N results limit reached. Use limit=M for more]'
  
  - All notices now in text content (LLM sees them)
  - TUI simplified (notices render as part of output)
  - Never return partial lines (except bash edge case)

  Mario Zechner · 2025-12-07 01:11:31 +01:00
  b813a8b92b
• Add totalTokens field to Usage type

  - Added totalTokens field to Usage interface in pi-ai
  - Anthropic: computed as input + output + cacheRead + cacheWrite
  - OpenAI/Google: uses native total_tokens/totalTokenCount
  - Fixed openai-completions to compute totalTokens when reasoning tokens present
  - Updated calculateContextTokens() to use totalTokens field
  - Added comprehensive test covering 13 providers
  
  fixes #130

  Mario Zechner · 2025-12-06 22:46:02 +01:00
  86e5a70ec4
• Add context overflow detection utilities

  Extract overflow detection logic into reusable utilities:
  - isContextOverflowError() to detect overflow from error messages
  - isContextOverflowFromUsage() to detect overflow from token usage
  - Patterns for Anthropic, OpenAI, Google, xAI, Groq, OpenRouter, llama.cpp, LM Studio
  
  Fixes #129

  Mario Zechner · 2025-12-06 21:24:15 +01:00
  a325c1c7d1
• Fix up prompt for image-tool-result for Gemini
  Mario Zechner · 2025-11-20 17:09:32 +01:00
  4afb3231e4
• Release v0.7.17
  Mario Zechner · 2025-11-18 17:49:12 +01:00
  a11c1aa4ff
• Add image support in tool results across all providers

  Tool results now use content blocks and can include both text and images.
  All providers (Anthropic, Google, OpenAI Completions, OpenAI Responses)
  correctly pass images from tool results to LLMs.
  
  - Update ToolResultMessage type to use content blocks
  - Add placeholder text for image-only tool results in Google/Anthropic
  - OpenAI providers send tool result + follow-up user message with images
  - Fix Anthropic JSON parsing for empty tool arguments
  - Add comprehensive tests for image-only and text+image tool results
  - Update README with tool result content blocks API

  Mario Zechner · 2025-11-12 10:45:56 +01:00
  84dcab219b
• Fix token statistics on abort for Anthropic provider

  - Add handling for message_start event to capture initial token usage
  - Fix message_delta to use assignment (=) instead of addition (+=)
    since Anthropic sends cumulative token counts, not incremental
  - Add comprehensive tests for all providers (Google, OpenAI Completions,
    OpenAI Responses, Anthropic)
  - Document OpenAI limitation: token stats only available at stream end
  
  Fixes issue where aborted streams had zero token counts despite
  Anthropic sending input tokens in the initial message_start event.

  Mario Zechner · 2025-10-26 21:22:24 +01:00
  bc8d994a7b
• Add timestamp to messages
  Mario Zechner · 2025-10-26 00:43:43 +02:00
  55dc0b6e08
• Add Unicode surrogate sanitization for all providers

  Fixes issue where unpaired Unicode surrogates in tool results cause JSON serialization errors in API providers, particularly Anthropic.
  
  - Add sanitizeSurrogates() utility function to remove unpaired surrogates
  - Apply sanitization in all provider convertMessages() functions:
    - User message text content (string and text blocks)
    - Assistant message text and thinking blocks
    - Tool result output
    - System prompts
  - Valid emoji (properly paired surrogates) are preserved
  - Add comprehensive test suite covering all 8 providers
  
  Previously only Google and Groq handled unpaired surrogates correctly.
  Now all providers (Anthropic, OpenAI Completions/Responses, Google, xAI, Groq, Cerebras, zAI) sanitize text before API submission.

  Mario Zechner · 2025-10-13 14:26:54 +02:00
  4e7a340460
• Add ToolRenderResult interface for custom tool rendering

  - Changed ToolRenderer return type from TemplateResult to ToolRenderResult
  - ToolRenderResult = { content: TemplateResult, isCustom: boolean }
  - isCustom: true = no card wrapper, false = wrap in card
  - Updated all existing tool renderers to return new format
  - Updated Messages.ts to handle custom rendering
  
  This enables tools to render without default card chrome when needed.

  Mario Zechner · 2025-10-11 04:40:42 +02:00
  b129154cc8
• Remove tool calls for which there are no results in subsequent user messages.
  Mario Zechner · 2025-10-01 22:18:30 +02:00
  51f5448a5c
• refactor(ai): improve error handling and stop reason types

  - Add 'aborted' as a distinct stop reason separate from 'error'
  - Change AssistantMessage.error to errorMessage for clarity
  - Update error event to include reason field ('error' | 'aborted')
  - Map provider-specific safety/refusal reasons to 'error' stop reason
  - Reorganize utility functions into utils/ directory
  - Rename agent.ts to agent-loop.ts for better clarity
  - Fix error handling in all providers to properly distinguish abort from error

  Mario Zechner · 2025-09-18 19:57:13 +02:00
  2296dc4052
• feat(ai): add partial JSON parsing for streaming tool calls

  - Added partial-json package for parsing incomplete JSON during streaming
  - Tool call arguments now contain partially parsed JSON during toolcall_delta events
  - Enables progressive UI updates (e.g., showing file paths before content is complete)
  - Arguments are always valid objects (minimum empty {}), never undefined
  - Full validation still occurs at toolcall_end when arguments are complete
  - Updated all providers (Anthropic, OpenAI Completions/Responses) to use parseStreamingJson
  - Added comprehensive documentation and examples in README
  - Added test to verify arguments are always defined during streaming

  Mario Zechner · 2025-09-16 12:23:34 +02:00
  39c626b6c9
• Fix NodeJS compat
  Mario Zechner · 2025-09-16 02:19:47 +02:00
  197259c88a
• Replace Zod with TypeBox for schema validation

  - Switch from Zod to TypeBox for tool parameter schemas
  - TypeBox schemas can be serialized/deserialized as JSON
  - Use AJV for runtime validation instead of Zod's parse
  - Add StringEnum helper for Google API compatibility (avoids anyOf/const patterns)
  - Export Type and Static from main package for convenience
  - Update all tests and documentation to reflect TypeBox usage

  Mario Zechner · 2025-09-16 01:10:40 +02:00
  e8370436d7
• feat(ai): Implement Zod-based tool validation and improve Agent API

  - Replace JSON Schema with Zod schemas for tool parameter definitions
  - Add runtime validation for all tool calls at provider level
  - Create shared validation module with detailed error formatting
  - Update Agent API with comprehensive event system
  - Add agent tests with calculator tool for multi-turn execution
  - Add abort test to verify proper handling of aborted requests
  - Update documentation with detailed event flow examples
  - Rename generate.ts to stream.ts for clarity

  Mario Zechner · 2025-09-09 14:58:54 +02:00
  35fe8f21e9
• Stop GPT-OSS 20b from being dumb ..
  Mario Zechner · 2025-09-09 04:31:09 +02:00
  594b0dac6c
• Fix streaming for z-ai in anthropic provider, add preliminary support for tool call streaming. Only reporting argument string deltas, not partial JSON objects
  Mario Zechner · 2025-09-09 04:26:56 +02:00
  98a876f3a0
• chore: bump version to 0.5.31
  Mario Zechner · 2025-09-07 00:09:34 +02:00
  2bdb87dfe7
• feat(ai): Add zAI provider support

  - Add 'zai' as a KnownProvider type
  - Add ZAI_API_KEY environment variable mapping
  - Generate 4 zAI models (glm-4.5-air, glm-4.5v, etc.) using anthropic-messages API
  - Add comprehensive test coverage for zAI provider in generate.test.ts and empty.test.ts
  - Models support reasoning/thinking capabilities and tool calling

  Mario Zechner · 2025-09-07 00:09:15 +02:00
  d073953ef7
• fix(ai): Sanitize tool call IDs for Anthropic API compatibility

  - Anthropic API requires tool call IDs to match pattern ^[a-zA-Z0-9_-]+$
  - OpenAI Responses API generates IDs with pipe character (|) which breaks Anthropic
  - Added sanitizeToolCallId() to replace invalid characters with underscores
  - Fixes cross-provider handoffs from OpenAI Responses to Anthropic
  - Added test to verify the fix works

  Mario Zechner · 2025-09-04 05:17:08 +02:00
  6679a83b32
• Clean-up
  Mario Zechner · 2025-09-03 00:01:32 +02:00
  acf0f5aee2
• Massive refactor of API

  - Switch to function based API
  - Anthropic SDK style async generator
  - Fully typed with escape hatches for custom models

  Mario Zechner · 2025-09-02 23:59:36 +02:00
  66cefb236e
• feat(ai): Add new streaming generate API with AsyncIterable interface

  - Implement QueuedGenerateStream class that extends AsyncIterable with finalMessage() method
  - Add new types: GenerateStream, GenerateOptions, GenerateOptionsUnified, GenerateFunction
  - Create generateAnthropic function-based implementation replacing class-based approach
  - Add comprehensive test suite for the new generate API
  - Support streaming events with text, thinking, and tool call deltas
  - Map ReasoningEffort to provider-specific options
  - Include apiKey in options instead of constructor parameter

  Mario Zechner · 2025-09-02 18:07:46 +02:00
  004de3c9d0
• test(ai): Add empty assistant message tests

  - Test providers handling empty assistant messages in conversation flow
  - Pattern: user message -> empty assistant -> user message
  - All providers handle empty assistant messages gracefully
  - Tests ensure providers can continue conversation after empty response

  Mario Zechner · 2025-09-02 02:10:07 +02:00
  be07c08a75
• test(ai): Add empty message tests for all providers

  - Test handling of empty content arrays
  - Test handling of empty string content
  - Test handling of whitespace-only content
  - All providers handle these edge cases gracefully

  Mario Zechner · 2025-09-02 02:03:06 +02:00
  0ac05a0676
• feat(ai): Fetch Anthropic, Google, and OpenAI models from models.dev instead of OpenRouter

  - Updated generate-models.ts to fetch these providers directly from models.dev API
  - OpenRouter now only used for xAI and other third-party providers
  - Fixed test model IDs to match new model names from models.dev
  - Removed unused import from google.ts

  Mario Zechner · 2025-09-02 01:18:59 +02:00
  efaa5cdb39
• feat(ai): Add cross-provider message handoff support

  - Add transformMessages utility to handle cross-provider compatibility
  - Convert thinking blocks to <thinking> tagged text when switching providers
  - Preserve native thinking blocks when staying with same provider/model
  - Add comprehensive handoff tests verifying all provider combinations
  - Fix OpenAI Completions to return partial results on abort
  - Update tool call ID format for Anthropic compatibility
  - Document cross-provider handoff capabilities in README

  Mario Zechner · 2025-09-01 18:43:49 +02:00
  46b5800d36
• refactor(ai): Update API to support partial results on abort

  - Anthropic, Google, and OpenAI Responses providers now return partial results when aborted
  - Restructured streaming to accumulate content blocks incrementally
  - Prevents submission of thinking/toolCall blocks from aborted completions in multi-turn conversations
  - Makes UI development easier by providing partial content even when requests are interrupted

  Mario Zechner · 2025-09-01 01:57:45 +02:00
  bf1f410c2b
• feat(ai): Add start event emission to all providers

  - Emit start event with model and provider info after creating stream
  - Add abort signal tests for all providers
  - Update README abort signal section to reflect non-throwing API
  - Fix model references in README examples

  Mario Zechner · 2025-08-31 23:09:14 +02:00
  a132b8140c
• fix(ai): Fix OpenAI Responses provider multi-turn conversation support

  - Collect complete output items during streaming instead of building blocks incrementally
  - Handle reasoning summary parts with proper newline separation
  - Support refusal content in message outputs
  - Preserve full reasoning items and message IDs for multi-turn resubmission
  - Emit proper streaming events for text and thinking deltas

  Mario Zechner · 2025-08-31 22:11:08 +02:00
  ee4c131873
• refactor(ai): Update OpenAI Completions provider to new content block API
  Mario Zechner · 2025-08-31 20:59:57 +02:00
  a72e6d08d4
• refactor(ai): Update API to support multiple thinking and text blocks

  BREAKING CHANGE: AssistantMessage now uses content array instead of separate fields
  - Changed AssistantMessage.content from string to array of content blocks
  - Removed separate thinking, toolCalls, and signature fields
  - Content blocks can be TextContent, ThinkingContent, or ToolCall types
  - Updated streaming events to include start/end events for text and thinking
  - Fixed multiTurn test to handle new content structure
  
  Note: Currently only Anthropic provider is updated to work with new API
  Other providers need to be updated to match the new interface

  Mario Zechner · 2025-08-31 19:32:12 +02:00
  f29752ac82
• test(ai): Add image input test for Anthropic Haiku 3.5

  - Added image handling test for Claude 3.5 Haiku
  - Ensures vision capabilities are properly tested

  Mario Zechner · 2025-08-30 23:00:30 +02:00
  f8a81b9453
• fix(ai): Fix OpenAI Responses provider multi-turn conversation support

  - Added contentSignature tracking for assistant messages
  - Fixed message format in convertToResponsesFormat (output_text instead of input_text)
  - Properly preserve message IDs for multi-turn conversations
  - Added proper ResponseOutputMessage type satisfaction
  - Updated tests to cover more providers and multi-turn scenarios

  Mario Zechner · 2025-08-30 22:55:29 +02:00
  cff766d3e2
• feat(ai): Add image input tests for vision-capable models

  - Added image tests to OpenAI Completions (gpt-4o-mini)
  - Added image tests to Anthropic (claude-sonnet-4-0)
  - Added image tests to Google (gemini-2.5-flash)
  - Tests verify models can process and describe the red circle test image

  Mario Zechner · 2025-08-30 18:37:17 +02:00
  796e48b80e
• refactor(ai): Update LLM implementations to use Model objects

  - LLM constructors now take Model objects instead of string IDs
  - Added provider field to AssistantMessage interface
  - Updated getModel function with type-safe model ID autocomplete
  - Fixed Anthropic model ID mapping for proper API aliases
  - Added baseUrl to Model interface for provider-specific endpoints
  - Updated all tests to use getModel for model instantiation
  - Removed deprecated models.json in favor of generated models

  Mario Zechner · 2025-08-30 00:21:03 +02:00
  f9d688d577