• LLMs power new apps but prompt injection is top OWASP threat. • Attack injects malicious instructions into untrusted data, overriding trusted prompts. • Real-world examples: Yelp reviews manipulated to favor specific restaurants. • StruQ and SecAlign are fine-tuning defenses that preserve utility. • StruQ drops optimization-free attack success to ~0% across LLMs. • SecAlign reduces optimization-based attacks to <15%, quadrupling prior state-of-the-art.

Article Summaries:

  • Researchers have introduced two fine‑tuning defenses-Structured Queries (StruQ) and Preference Optimization (SecAlign)-to counter prompt‑injection attacks on large language models (LLMs). Prompt injection, identified by OWASP as the top threat to LLM‑integrated applications, occurs when untrusted data contains hidden instructions that override trusted prompts. StruQ trains LLMs to ignore injected commands by simulating such attacks during fine‑tuning, while SecAlign uses preference optimization to strongly favor correct responses over injected ones. Together, the methods reduce attack success rates to near zero for many optimization‑free attacks and cut strong optimization‑based attacks below 15%, outperforming prior state‑of‑the‑art defenses across five tested models.

Sources: