Study finds SynthID text watermarking can alter model refusals and agent tool calls

The study, titled The Provenance Tax and written by Lasso researcher Andrea Siposova, tested Phi-4, Llama-3.1-8B, Granite-3.2-8B, two Gemma 3 models and two Qwen3 models. Watermarking embeds a statistical pattern by nudging which tokens a model picks. The author calls the side effect sampling drift: the weights and prompt stay the same, but different tokens are chosen, and in an agent those tokens decide which tool is called and with what arguments.

On the BFCL v4 tool-calling benchmark, watermarked and unwatermarked runs disagreed in 6.5 percent of cases on average across 21 combinations of model and temperature, according to Unite.AI. Refusals of plainly harmful requests barely moved. Under a prompt-injection attack, compliance rose 12.5 points for Gemma-3-27b and 9 points for Gemma-3-12b, while results across watermark keys ranged from a 4.5-point drop to a 14.5-point rise in attack success.

Lasso does not argue against watermarking. It recommends running security evaluations and red-teaming with the exact watermark configuration used in deployment. The work comes from a security vendor, covers only SynthID-Text, and the coverage reviewed included no response from Google.