Commit Graph

1 Commits

  • feat: context compaction (#3446)
    ## Compact feature:
    1. Stops the model when the context window become too large
    2. Add a user turn, asking for the model to summarize
    3. Build a bridge that contains all the previous user message + the
    summary. Rendered from a template
    4. Start sampling again from a clean conversation with only that bridge