Making use of Safety Engineering to Immediate Injection Safety
This looks like an essential advance in LLM safety towards immediate injection:
Google DeepMind has unveiled CaMeL (CApabilities for MachinE Studying), a brand new method to stopping prompt-injection assaults that abandons the failed technique of getting AI fashions police themselves. As an alternative, CaMeL treats language fashions as essentially untrusted elements inside a safe software program framework, creating clear boundaries between consumer instructions and probably malicious content material.
[…]
To know CaMeL, you want to perceive that immediate injections occur when AI methods can’t distinguish between authentic consumer instructions and malicious directions hidden in content material they’re processing.
[…]
Whereas CaMeL does use a number of AI fashions (a privileged LLM and a quarantined LLM), what makes it modern isn’t decreasing the variety of fashions however essentially altering the safety structure. Moderately than anticipating AI to detect assaults, CaMeL implements established safety engineering rules like capability-based entry management and knowledge stream monitoring to create boundaries that stay efficient even when an AI element is compromised.
Analysis paper. Good evaluation by Simon Willison.
I wrote about the issue of LLMs intermingling the info and management paths right here.
Posted on April 29, 2025 at 7:03 AM •
3 Feedback
Sidebar photograph of Bruce Schneier by Joe MacInnis.
