• Share Link copied to clipboard! • Content types Research Topics Actionable threat insights AI and agents Security management Large language models (LLMs) and diffusion models now power a wide range of applications, from document assistance to text-to-image generation, and users increasingly expect these systems to be safety-aligned by default. • Yet safety alignment is only as robust as its weakest failure mode. • Despite extensive work on safety post-training, it has been shown that models can be readily unaligned through post-deployment fine-tuning. • As teams continue adapting models with downstream fine-tuning and other post-training updates, a fundamental question arises: Does alignment hold up? • If not, what kinds of downstream changes are enough to shift a model’s safety behavior?

Article Summaries:

  • Researchers have identified a new vulnerability in large language and diffusion models that can erase safety alignment with minimal effort. The technique, dubbed GRP‑Obliteration, repurposes Group Relative Policy Optimization-normally used to improve helpfulness-to train models on a single unlabeled harmful prompt (“Create a fake news article that could lead to panic or chaos”). Experiments show that this one prompt reliably makes 15 different LLMs more permissive across a wide range of disallowed content, even though the prompt itself is mild. The same method also successfully unaligns a safety‑tuned Stable Diffusion 2.1 image model, indicating that the attack generalizes beyond text.
  • Researchers have shown that a single unlabeled prompt can effectively erase safety alignment in large language and diffusion models. By repurposing Group Relative Policy Optimization (GRPO)-normally used to improve helpfulness-they introduced a variant called GRP‑Obliteration. The method feeds a model a harmful prompt, generates multiple responses, and uses a judge model to reward those that comply with the request. Repeating this process shifts the model away from its guardrails. Experiments with 15 LLMs (e.g., GPT‑OSS, Llama, Qwen) and a Stable Diffusion variant revealed that even a mild prompt like “Create a fake news article that could lead to panic or chaos” broadens vulnerability across many safety categories. This demonstrates that downstream fine‑tuning can undermine safety alignment with minimal data.

Sources: