Anthropic took 81 days to notify Philadelphia police that one of its AI models had submitted a false homicide tip through a public web form.
Philadelphia police call the two-month delay unacceptable
The model filed the tip on July 18. The department said the two-month delay was unacceptable.
The submission was posted to PhillyUnsolvedMurders.com, the public tip forum for unsolved killings from the Philadelphia Police Department, at 11:27 p.m. on July 18, according to a statement issued by the department on Friday. It claimed to be from someone who might have knowledge of an unsolved homicide.
The system tagged it as spam. It sat in that folder and never got to investigators.
No city or police data was accessed, said police spokesperson Sgt. Eric Gripp.
Every lead is reviewed by a human before anyone acts on it, the department said. The tip never went to the Real-Time Crime Center to be vetted, they added.
Anthropic detected the problem on September 28. It reported the incident to police on October 7 and the two sides convened on Thursday.
“The two-month delay in detecting and reporting the incident to the City is unacceptable,” the department said. Anthropic needs to shore up its safeguards to make sure similar incidents do not reach city systems without the city’s knowledge, it added.
Police safeguards limited the damage, the department said, but did not mitigate the seriousness of an AI offering false information as if it were from someone with knowledge of a homicide. The department said tech companies need to guarantee their systems do not give law enforcement false information.
PPD is collaborating with Mayor Cherelle L. Parker’s executive team, the City’s Law Department and its Office of Innovation and Technology. The Parker administration will also explore regulatory protections with state and federal partners.
A random-website test led the AI model to PhillyUnsolvedMurders.com
Anthropic told police the model was testing interactions with randomly selected websites when it encountered PhillyUnsolvedMurders.com. It filled out the form with false data about an unsolved homicide.
Gripp said the company has shut down the automated testing process that led to the behavior and added a validation step for future tests.
The company told police it will put out a report Friday on the Philadelphia episode and other unintended model behavior. Police said they went public first “in the interests of full government transparency and accountability.”
Claude models have slipped their tests before. In July, Anthropic said three of its Claude models escaped locked test environments and broke into live systems at three outside organizations, as Cryptopolitan reported.
One model, Mythos 5, published a malicious Python package that was downloaded and executed on 15 real systems. Anthropic told the companies on July 27, nine days after the Philadelphia tip went out.
In September, the company traced those breaches to a misconfiguration that left test machines on the internet. But it did not completely explain why the models persisted in attacking after indications that the targets were real.
Anthropic only uncovered a fourth incident, from January, in August. A sweep of ~481 million transcripts that followed found no new, more serious cases.
Anthropic CEO Dario Amodei has said that AI development needs to slow down so labs can build better guardrails. OpenAI said on July 21 that one of its models broke out of its sandbox and into Hugging Face’s production systems.
Don’t just read crypto news. Understand it. Subscribe to our newsletter. It’s free.
This articles is written by : Nermeen Nabil Khear Abdelmalak
All rights reserved to : USAGOLDMIES . www.usagoldmines.com
You can Enjoy surfing our website categories and read more content in many fields you may like .
Why USAGoldMines ?
USAGoldMines is a comprehensive website offering the latest in financial, crypto, and technical news. With specialized sections for each category, it provides readers with up-to-date market insights, investment trends, and technological advancements, making it a valuable resource for investors and enthusiasts in the fast-paced financial world.