wirjo a5fac1ef09 feat(amazon-bedrock): conditionally omit maxTokens from inferenceConfig (#3400) ...

When model.maxTokens is 0 (unset/unknown), omit maxTokens from the
Bedrock ConverseStream inferenceConfig instead of sending 0.

Bedrock's inferenceConfig.maxTokens is optional — when omitted, the
model uses its own internal default. This is optimal for Bedrock's
token quota management:

- At request start, Bedrock reserves input_tokens + max_tokens from
  your TPM quota
- For Claude 3.7+ models, output tokens have a 5x burndown rate
- Sending an unnecessarily high maxTokens wastes TPM capacity during
  the reservation window, reducing concurrent request throughput
- Omitting it lets Bedrock use the model default (~4096 for Claude),
  matching expected output sizes and maximizing quota utilization

Also applies the same conditional pattern to temperature — only
include it in inferenceConfig when explicitly set.

Changes:
- simple-options.ts: Use ?? instead of || so explicit 0 is preserved;
  return undefined when model.maxTokens is 0 (unset)
- amazon-bedrock.ts: Spread maxTokens and temperature into
  inferenceConfig only when defined

Fixes #3399

a5fac1ef09 · 2026-04-19 08:48:43 +02:00

History

..

agent

Add [Unreleased] section for next cycle

2026-04-18 01:32:50 +02:00

ai

feat(amazon-bedrock): conditionally omit maxTokens from inferenceConfig (#3400)

2026-04-19 08:48:43 +02:00

coding-agent

fix(coding-agent): coerce stringified JSON edits in edit tool (#3370)

2026-04-18 18:29:23 +02:00

mom

Add [Unreleased] section for next cycle

2026-04-18 01:32:50 +02:00

pods

Release v0.67.68

2026-04-18 01:31:38 +02:00

tui

Add [Unreleased] section for next cycle

2026-04-18 01:32:50 +02:00

web-ui

Add [Unreleased] section for next cycle

2026-04-18 01:32:50 +02:00