What is a recommended approach to preserve essential context in a constrained memory setting for a chat prompt?

• A. Repeat the entire conversation in every turn.
• B. Summarize the state after each turn and store memory externally.
• C. Remove all context to save tokens.
• D. Rely on user to remind the model of prior topics.

Answer: (B)

In constrained memory chat prompts, keeping a compact, retrievable representation of the prior conversation is essential. The best approach is to summarize the state after each turn and store memory externally. Summaries capture the important elements—topics discussed, user goals, key facts, and decisions—in a short, token-efficient form that the model can reference. Storing this externally (in a memory store or vector database) lets you fetch and reconstruct relevant context for new turns without loading the full history into the prompt. This enables continuity across long chats even when the model’s token window is limited, and it makes it possible to update or prune memory as conversations evolve. Practically, after each message, generate a concise summary and attach relevant metadata, then retrieve the stored memory at the start of the next turn to re-inform the prompt.

Repeating the entire conversation wastes tokens and quickly exhausts the limit. Removing all context loses necessary continuity. Relying on the user to remind you of prior topics is unreliable and disrupts flow.