AIR

Context optimization

AIR can shorten what it sends to a provider for a long conversation, while keeping the full original history intact in the session store (see core/sessions.md).

When it runs

Every call to air.chat(...) runs context optimization first, unless AIRConfig.context_optimization == "off". This includes calls made through Session.chat(...), since that method calls air.chat(...) internally with the combined history.

Configuration

python
from air.config import AIRConfig
from air.context.optimize import ContextBudget
 
air = AIR(config=AIRConfig(
    context_optimization="auto",   # or "off"
    context_budget=ContextBudget(max_tokens=8000, keep_recent=6, max_message_chars=4000),
))

ContextBudget fields:

  • max_tokens (default 8000): if the estimated token count of the conversation is at or under this, and/or there are keep_recent or fewer messages, no summarization happens (see below; oversized individual messages can still be trimmed).
  • keep_recent (default 6): number of most recent non-system messages preserved exactly, without summarization, when compression does happen.
  • max_message_chars (default 4000): any single message longer than this is trimmed (head and tail kept, middle replaced with a ...[truncated, N chars omitted]... marker), independent of whether summarization happens. This covers very large individual messages, such as a large pasted tool output.

Invalid values for AIRConfig.context_optimization (anything other than "off" or "auto") raise ConfigurationError when constructing AIR(...).

What happens when compression triggers

Compression (needs_compression in air.context.optimize) triggers only when both are true: the conversation's estimated tokens exceed max_tokens, and there are more messages than keep_recent. When it triggers:

  1. System messages are kept as is.
  2. The most recent keep_recent non-system messages are kept exactly (after per-message trimming if any exceed max_message_chars).
  3. Everything else is replaced by a single system message: "Summary of earlier conversation: {summary}".

When compression does not trigger, individual oversized messages are still trimmed to max_message_chars, but nothing is summarized and no messages are dropped.

Summarization strategy

By default, air.context.optimize.local_summary builds the summary with no AI call: it joins the older messages as "role: content | role: content | ..." and truncates the joined text to 500 characters if needed. This is a compaction, not a semantic summary; it does not attempt to preserve meaning beyond truncation.

AI assisted summarization (opt in)

python
air = AIR(config=AIRConfig(summarize_with_ai=True, summarize_provider="groq"))

If summarize_with_ai=True and at least one provider is registered, AIR asks a provider directly (bypassing routing and fallback entirely, calling that provider's chat(...) method directly) to summarize the older messages, using summarize_provider if set, or the highest priority registered provider otherwise. If that call raises any exception, AIR falls back to the local, non-AI summary instead. This calls a provider and spends its quota; it is off by default so AIR never does this without being asked.

What gets sent vs. what stays stored

The optimized (possibly summarized and trimmed) message list is what gets sent to the provider for that one request. It is never written back to the session store; the original full history in the store is untouched. Each Session.chat(...) call re-runs optimization fresh against the full stored history plus the new message.

Optimization metadata on the response

python
response = await air.chat(request)
print(response.original_estimated_tokens)
print(response.optimized_estimated_tokens)
print(response.estimated_tokens_saved)
print(response.compression_applied)
  • original_estimated_tokens: estimated tokens of the conversation before optimization.
  • optimized_estimated_tokens: estimated tokens of what was actually sent.
  • estimated_tokens_saved: the difference (never negative).
  • compression_applied: True only when summarization happened (older messages were replaced by a summary), not merely because an oversized message was trimmed.

These are estimates from AIR's character based token approximation, not a verified count of tokens actually billed by any provider, and not a claim of semantic accuracy for the summary.