Long AI agent conversations burn tokens fast: 10 rounds can eat thousands of tokens, and 1000 sessions a day can cost tens of dollars. Message compression cuts the history payload by half or more while keeping key information. Here are 4 implementable methods with code.

💡 What You Will Learn

Long AI agent conversations burn tokens fast: 10 rounds can eat thousands of tokens, and 1000 sessions a day can cost tens of dollars. Message compression cuts the history payload by half or more whil

📜 Table of Contents

• Method 1: Sliding Window
• Method 2: History Summarization
• Method 3: Key-Info Extraction
• Method 4: Hybrid — Recent Full, Middle Summarized, Early Extracted
• Combined Effect
• Checklist
• FAQ

AI Agent Message Compression: 4 Methods to Cut Token Costs Without Losing Key Info

The math: an agent handling 10 rounds at ~2000 tokens per round, 1000 conversations a day, means 20M tokens — tens of dollars daily at flagship model rates. Message compression shrinks the history sent to the model by half or more. Four methods, from simplest to most sophisticated, each with code.

Method 1: Sliding Window

Keep only the last N rounds; drop everything older. Zero extra LLM calls, 5 lines of code, never blows the context window. Downside: early information (preferences, agreements) is lost.

def sliding_window(conversation, keep_rounds=5):
    return conversation[-(keep_rounds * 2):]

Fix: before dropping, extract each round's core user request into a one-line "requirements list" placed in the system prompt.

Method 2: History Summarization

Every N rounds, compress finished turns into a summary with one cheap model call.

def summarize_rounds(conversation, llm, compress_every=5):
    if len(conversation) < compress_every * 2:
        return conversation
    done, active = conversation[:-(compress_every * 2)], conversation[-(compress_every * 2):]
    summary = llm.summarize(done)
    return [{"role": "system", "content": f"Summary: {summary}"}] + active

60-70% compression with controlled loss. Fix: force a structured 4-section summary template — user goal / confirmed info / open items / constraints — so the summary stays queryable.

Method 3: Key-Info Extraction

Keep only three things: user intent, facts gathered, todos. Extract as JSON with a small model; store results in memory (vector DB or KV) and retrieve on demand instead of stuffing every request.

EXTRACT_PROMPT = """Extract structured info as JSON only:
{"intent": "...", "facts": ["..."], "todos": ["..."]}
Conversation: {conversation}"""

70-85% compression, minimal loss, but costs one call and depends on model understanding.

Method 4: Hybrid — Recent Full, Middle Summarized, Early Extracted

Keep the last 3 rounds verbatim, summarize the middle, extract key info from the early part, each with a token budget (e.g. 60/30/10%). Best quality-to-cost ratio; tune the ratios with real traffic over a week.

Combined Effect

(Compression = token reduction vs raw history; actuals vary.)

Checklist

1. Compress only when context exceeds budget.
2. Use cheap small models for summarize/extract; flagship only for the final answer.
3. Tag compressed messages for quality review.
4. Run sensitive data through redaction before injecting.

FAQ

Q: Does compression cause "amnesia"? A: Partially. Keep must-remember items (preferences, hard constraints) in the system prompt or memory, not the history.

Q: Isn't summarizing itself costly? A: One cheap call of a few hundred tokens replaces thousands of tokens paid every round — net 50%+ savings on long sessions.

Q: Chinese conversations? A: Chinese has higher token density, so compression gains are usually larger; use Chinese-strong summarizers (Qwen, DeepSeek).

Q: Vector memory vs compression? A: Vector memory handles cross-session recall of old facts; compression controls in-session context length. They complement each other.

Q: Existing libraries? A: LangChain and LlamaIndex ship built-in summarization/compression memory components; or implement the templates above yourself.

Token prices vary by provider and version — check official pricing pages.