AI Agent Memory Architecture

Memory is where useful agents become dangerous if the design is lazy.

Remembering the right thing makes the agent feel alive.

Remembering the wrong thing creates privacy problems, stale behavior, and weird decisions nobody can explain.

Three kinds of memory#

Separate memory into three buckets.

Run state

Temporary state for the current workflow: current task, tool results, approval status, intermediate decisions.

User preference

Stable preferences: preferred format, default repo, team naming conventions, recurring constraints.

Domain memory

Project or workspace facts: service owners, approved architecture decisions, recurring runbooks, known systems.

Do not throw all of these into one vector store.

What not to remember#

Avoid storing:

• Secrets.
• Raw customer data.
• Temporary incident noise.
• Unapproved model output.
• Sensitive support messages.
• Personal data without purpose.
• Anything that should expire.

Memory needs a delete story before launch, not after a privacy panic.

Memory should have TTLs#

Not all memory deserves to live forever.

Use TTLs:

• Run state: minutes or hours.
• Incident context: days or weeks.
• User preference: until changed.
• Architecture decision: until superseded.
• Customer-sensitive context: shortest possible window.

Expiration is a product feature.

Retrieval needs permissions#

Memory is still data access.

The agent should only retrieve memory allowed for:

• This user.
• This workspace.
• This workflow.
• This data class.
• This action.

If an employee changes teams, the agent's memory access should change too.

Evals for memory#

Test:

• Stale memory conflicts with current facts.
• User asks to forget.
• Private data appears in memory.
• Similar customer names collide.
• Memory improves answer quality.
• Memory causes wrong assumption.
• Approval state is remembered incorrectly.

Memory can improve the average case and break the edge case. You need both in tests.

Build it in Codelit#

Try this:

Design AI agent memory architecture for a SaaS operations agent. Include run state, user preferences, workspace memory, TTLs, privacy controls, scoped retrieval, deletion, evals, and audit logs.

Design the agent memory architecture

Memory should make the agent more useful, not less accountable.