Claude text watermarks will “nudge” its word choices. Should we care? | usagoldmines.com

Soon, Claude will be watermarking all its text outputs, and those watermarks will be invisible, Anthropic has been telling us. The big question, of course, is whether those watermarks will change the meaning — or lower the quality — of what Claude writes.

Anthropic gave a nuanced answer to that question in a recently published blog post, arguing that the watermarking method it’s chosen “does not impact the quality of Claude’s output,” although it will nevertheless “nudge” some of the model’s word choices.

“To a reader, a watermarked response is indistinguishable from an unwatermarked one,” Anthropic said. “In internal testing, we’ve seen no impact of watermarking on the content, level of creativity, or readability of Claude’s text.”

Claude’s text watermarks, which are coming in response to the recently adopted EU AI Act, will employ a “version” of Google DeepMind’s SynthID-Text process, which “changes the source of the randomness used to pick among words.”

As Anthropic explains, Claude’s method of writing is similar to other LLMs: It generates each word one at a time, calculating the probability of each subsequent word and then picking from among the most likely choices.

In some cases, such as responses with factual information, there may only be one good choice for a given word. For example, if you ask Claude who was the first human on the moon, there will be an obvious best choice for the next word after “Neil.” Similarly, ask Claude what 2 + 2 is, and “4” will be the “very clear best choice” for the next word, Anthropic says.

But (in an example served up by Anthropic), when generating a sentence about a cloudy weather forecast, Claude might have a range of likely next words in the sentence “It’s going to be a…” The word “grey” could be a top choice with a 30-percent probability (I’m making that percentage up for argument’s sake), as well as “overcast” with a 28-percent probability.

Even without watermarking, Claude won’t necessarily pick the word “gray” just because it has a higher probability ranking than “overcast.” In a close contest like this one, the word Claude eventually chooses comes down to a roll of the dice.

It’s in close (and “low-stakes”) cases like those where Claude watermarking would come into play, with the watermarking process giving a “nudge” in one direction or another, according to Anthropic. That nudge isn’t enough to change a Claude answer to, say, “Neil Smith” for the “who took the first steps on the moon” question, but it could affect whether Clause says it’s a “gray” day or an “overcast” one. 

And while Claude won’t (or shouldn’t) alter a critical bit of code for a watermark, it might tweak a word choice in “low-stakes” portion, such as a comment in the code, Anthropic says.

Given a large enough sample, those subtly different word choices can become detectable as Claude watermarks, and those watermarks can survive a copy-paste and even “light editing,” Anthropic says. Users will be able to spot those watermarks using a “detection API” that’s still in the works, the company noted.

So, back to the original question: Do we care that Claude’s watermarks will affect its word choices, even if the “nudges” are subtle ones?

Apple observer and Daring Fireball writer John Gruber, for one, took Anthropic to task for the new watermarking policy, “It’s unacceptable for a tool to sacrifice an iota of clarity, coherence, meaning, quality, etc. for the purpose of embedding hidden clues within the text to suggest its provenance,” Gruber writes. “The idea that anything other than my needs should factor into the generation of text for me is patently offensive.”

Personally, I agree with Gruber’s sentiment: I wouldn’t want Anthropic throttling my Claude output quality so it can comply with a regulatory mandate. But will Claude watermarks actually make its text generation worse — or worse than it already is?

The whole issue of watermarking does make us look closer at how the sausage is made in terms of AI text generation, and it’s not pretty. Watermarks or no, the manner in which AI synthesizes text and samples its word choices are what makes AI writing sound like AI, and it’s also what leads to mistakes.

Consider this: While humans may agonize over the next word, weighing the meaning, the nuance, and even the cadence of the syllables, Claude, ChatGPT, and Gemini are instead poring over probabilities before ultimately making a random choice among the likeliest options. Put another way, human writers sweat the details, while AI models simply roll the dice.

We’ll have to wait for Claude watermarking to kick into gear before we can really assess how those watermarks affect the output. (Anthropic says it’s in the process of adding watermarking support to new and existing Claude models.) 

But if AI watermarking nudges Claude’s roll toward “gray” rather than “overcast,” did that watermark make the output worse, or just a different shade of meh? We’ll find out soon enough.