AI Memory, Mapped

Natalie Isak
Natalie Isak
ML Engineer, Microsoft
Abstract

Persistent memory transforms AI systems from stateless tools into context-aware systems, but it also introduces a class of risks. We will cover key risks including continuous exfiltration via prompt injection, delayed tool invocation, and negative psychosocial impacts. The second half focuses on building memory-safe systems by design: threat modeling memory, observability strategies, and runtime safety monitoring at scale (including BinaryShield, a novel privacy-preserving method for sharing textual customer content to detect coordinated spray attacks).

00:00:00 Introduction and talk overview

00:01:19 Defining AI memory and indirect prompt injection

00:03:08 What gets stored in AI memory

00:05:00 How memory changes system behavior

00:06:04 Types of AI memory

00:07:31 Memory in an agentic AI system

00:13:21 Memory poisoning and downstream risks

00:16:40 Attack surfaces across memory types

00:19:05 Parametric memory poisoning case study

00:22:12 Textual memory poisoning case study

00:27:00 Best design practices for AI memory

00:30:00 Secure inputs and structured storage

00:33:00 Access controls, UX, and sensitive data

00:39:00 Evaluation, red teaming, and monitoring

00:46:00 Cross-boundary detection with Binary Shield

01:00:00 Best practices recap and future work