Skip to main content

The problem with permanent memory

A memory system that never forgets will eventually degrade. Stale facts accumulate at equal weight to recent ones. The agent’s knowledge doesn’t mature — it just grows noisier. Engram implements a lifecycle where memories decay, consolidate, and evolve — modelled on cognitive science interference theory.

Decay

Memories decay when unused. The base formula:
new_confidence = current_confidence × (1 - decay_rate × effective_multiplier)
The effective_multiplier is where Engram’s competition-aware decay comes in.

Competition-aware decay

Semantically similar memories suppress each other’s confidence over time — inspired by interference theory from cognitive psychology. Two memories about “programming language preference” compete:
competition_factor = Σ(confidence_delta × similarity) / (1 + confidence)
effective_decay = base_decay_rate × (1 + competition_factor)
This means:
  • Memories in crowded semantic neighbourhoods decay faster
  • Distinct, unique memories decay more slowly
  • The agent’s knowledge naturally consolidates toward the most reinforced beliefs

Consolidation pipeline

The 5-stage background consolidation pipeline runs periodically:
Episodes → Semantic beliefs → Procedural skills → Schemas
1

Episode clustering

Raw episodic memories are grouped by semantic similarity into coherent experience clusters.
2

Belief extraction

Repeated patterns in episode clusters are extracted as semantic beliefs with confidence derived from cluster size and consistency.
3

Procedure detection

Successful trigger-action patterns from episodes are promoted to procedural memories.
4

Schema derivation

Dense clusters of consistent semantic memories are promoted to schema memories — high-level mental models of the user.
5

Archival

Memories below the archive threshold (confidence < 0.40) are soft-deleted and become recoverable but inactive.

Triggering decay manually

# Trigger decay for an agent (useful after long periods of inactivity)
result = client.cognitive.decay(agent_id=agent.id)
print(f"Decayed: {result.decayed_count} memories")
print(f"Archived: {result.archived_count} memories")

Triggering consolidation

# Run consolidation on recent episodes
result = client.cognitive.consolidate(
    agent_id=agent.id,
    scope="recent",  # or "full"
)

Restoring archived memories

Archived memories are soft-deleted — they can be restored:
# Restore an archived memory
client.memories.store(...)  # archived memories can be found via /v1/memories/recall with include_archived=true

Memory health monitoring

health = client.cognitive.health()

print(f"Overall health score: {health.overall_score:.2f}")
print(f"Confidence distribution:")
print(f"  Hot (>0.85):    {health.tier_distribution.hot}")
print(f"  Warm (0.70-0.85): {health.tier_distribution.warm}")
print(f"  Cold (0.40-0.70): {health.tier_distribution.cold}")
print(f"  Archive (<0.40):  {health.tier_distribution.archive}")
print(f"Active contradictions: {health.contradiction_count}")
print(f"Stale memories (30d):  {health.stale_count}")