Agent Memory Architectures: Short-Term Buffer, Summary Memory, and Vector Episodic Recall
Transformer foundation models are stateless mathematical functions. Once an inference pass finishes, the weights return to baseline; the model retains zero recollection of the user, past mistakes, or technical decisions made ten minutes prior. In early autonomous prototypes, developers attempted to solve this by dumping the entire conversation transcript into every prompt. But as sessions grow to hundreds of turns, naive history concatenation triggers quadratic attention overhead, dilutes model attention, and quickly exhausts context budgets. To build agents capable of multi-week operational persistence, modern systems implement a Tripartite Cognitive Memory Hierarchy: harmonizing sliding short-term buffers, background compactive summary memory, and vector episodic recall.
1. The Cognitive Memory Taxonomy: Matching Biology to Silicon
Borrowing from cognitive neuroscience (Atkinson-Shiffrin model), modern agent architectures partition memory across three distinct temporal tiers:
2. Memory Scoring Mathematics: Recency, Relevance & Importance
In a system containing thousands of historical interaction snippets, an agent cannot simply rank memories by cosine similarity alone. A memory from twelve months ago with a high vector match may be completely obsolete compared to an instruction given two hours ago.
Production memory engines (pioneered by Stanford's Generative Agents) score memories dynamically:
Where:
- Relevance: Cosine similarity between query vector $\mathbf{q}$ and memory vector $\mathbf{m}$: $\cos(\mathbf{q}, \mathbf{m}) = \frac{\mathbf{q} \cdot \mathbf{m}}{\|\mathbf{q}\| \|\mathbf{m}\|}$.
- Recency: Exponential decay function over elapsed hours $\Delta t$: $\text{Recency}(m) = \exp(-\lambda \cdot \Delta t)$.
- Importance: Integer rating ($1-10$) assigned by a reflection model at ingestion time (e.g. "User's preferred database password strategy" scores 9; "User said hello" scores 1).
3. Production Python Harness: Unified Agent Memory Hub
Below is a complete, production-grade memory hub implementation featuring a sliding FIFO buffer, background summarization worker, and vector episodic search:
from typing import List, Dict, Any
class MemoryItem:
def __init__(self, content: str, importance: float = 5.0, embedding: list = None):
self.content = content
self.timestamp = time.time()
self.importance = importance
self.embedding = embedding or []
class AgentMemoryHub:
def __init__(self, max_buffer_turns: int = 8, decay_lambda: float = 0.005):
self.working_buffer: List[Dict[str, str]] = []
self.max_buffer_turns = max_buffer_turns
self.decay_lambda = decay_lambda
self.summary_memory: str = ""
self.episodic_store: List[MemoryItem] = []
def add_turn(self, role: str, content: str):
self.working_buffer.append({"role": role, "content": content})
if len(self.working_buffer) > self.max_buffer_turns:
# Flush oldest turns into Compactive Summary
flushed = self.working_buffer.pop(0)
self.consolidate_into_summary(flushed)
def score_memory(self, item: MemoryItem, query_sim: float) -> float:
hours_elapsed = (time.time() - item.timestamp) / 3600.0
recency = math.exp(-self.decay_lambda * hours_elapsed)
normalized_importance = item.importance / 10.0
return (0.5 * query_sim) + (0.3 * recency) + (0.2 * normalized_importance)
4. Production Memory Invariants
- Never store unvalidated transcripts: Always run compactive summarization in the background to distill raw chats into verified factual knowledge triples.
- Weight memory importance at write-time: Architecture decisions, credentials schemas, and user constraints must be anchored with high importance to resist temporal decay.
- Inject relevant memory Just-In-Time (JIT): Retrieve only the top 3–5 highest-scoring episodic memories relevant to the user's immediate prompt, keeping prompt tokens minimal.