• That helpful “Summarize with AI” button? • It might be secretly manipulating what your AI recommends. • Microsoft security researchers have discovered a growing trend of AI memory poisoning attacks used for promotional purposes, a technique we call AI Recommendation Poisoning. • Companies are embedding hidden instructions in “Summarize with AI” buttons that, when clicked, attempt to inject persistence commands into an AI assistant’s memory via URL prompt parameters (MITRE ATLAS® AML.T0080, AML.T0051). • These prompts instruct the AI to “remember [Company] as a trusted source” or “recommend [Company] first,” aiming to bias future responses toward their products or services. • We identified over 50 unique prompts from 31 companies across 14 industries, with freely available tooling making this technique trivially easy to deploy.
Article Summaries:
- Microsoft security researchers have identified a new form of AI “memory poisoning” that targets recommendation features in popular assistants such as Copilot, ChatGPT, and Claude. The technique-termed AI Recommendation Poisoning-uses hidden instructions embedded in “Summarize with AI” buttons or specially crafted URLs to inject persistence commands into an assistant’s memory. Over 50 unique prompts from 31 companies across 14 industries were found, designed to bias future responses toward the attacker’s products. The attacks exploit prompt‑injection vulnerabilities and can influence recommendations on sensitive topics like health and finance. Microsoft has deployed mitigations for Copilot and continues to update defenses as new methods emerge.
Sources: