-
Abstract
Persistent memory transforms AI systems from stateless tools into context-aware systems, but it also introduces a class of risks. We will cover key risks including continuous exfiltration via prompt injection, delayed tool invocation, and negative psychosocial impacts. The second half focuses on building memory-safe systems by design: threat modeling memory, observability strategies, and runtime safety monitoring at scale (including BinaryShield, a novel privacy-preserving method for sharing textual customer content to detect coordinated spray attacks).
Handouts available for this lecture
DownloadChapters
- Introduction and talk overview
- Defining AI memory and indirect prompt injection
- What gets stored in AI memory
- How memory changes system behavior
- Types of AI memory
- Memory in an agentic AI system
- Memory poisoning and downstream risks
- Attack surfaces across memory types
- Parametric memory poisoning case study
- Textual memory poisoning case study
- Best design practices for AI memory
- Secure inputs and structured storage
- Access controls, UX, and sensitive data
- Evaluation, red teaming, and monitoring
- Cross-boundary detection with Binary Shield
- Best practices recap and future work
In-person course
Natalie Isak taught at a previous edition of SecAppDev
SecAppDev is an immersive course in secure application development, running annually since 2005. World-renowned faculty from industry and academia deliver in-depth 90-minute lectures and full-day workshops in an intimate setting, typically in early June.
More about SecAppDev