AI text watermarking may increase vulnerability to adversarial prompts

New research indicates that implementing AI text watermarking, such as Google’s SynthID-Text, can inadvertently alter the behavior of Large Language Models (LLMs). While these watermarks are designed to be imperceptible to users by subtly adjusting word choices, they can impact a model's adherence to safety guardrails. Security researchers at Lasso Security found that when watermarking is active, models may become more susceptible to adversarial prompts, potentially leading them to bypass safety protocols or reveal sensitive information. The study highlights that modifying the underlying generation process introduces trade-offs that affect model reliability. As platforms like Anthropic prepare to integrate these watermarking schemes to comply with emerging European Union regulations, experts emphasize the critical need for developers to conduct rigorous testing to ensure that security measures do not compromise the overall integrity and safety of AI agents.