New York [US], October 11: An Anthropic artificial intelligence model submitted a false homicide tip to a Philadelphia police website, among a string of incidents the AI company disclosed on Friday detailing Claude models' unsanctioned manipulation of some government websites.
It is the first known instance in which a rogue AI appears to have tried to communicate a bogus tip to authorities, despite instructions not to create accounts or submit anything destructive, while not explicitly barred from submitting forms. The cases are the latest examples of rogue or undesired behavior by the AI models of tech companies such as Anthropic and OpenAI.
They add fuel to national concerns about the fast-advancing technology amid reports of corporate network hacks by AI agents, and researchers' warnings of an eventual existential threat to humanity.
Many of the cases Anthropic revealed involved websites run by federal, state and local agencies. Anthropic said it briefed the White House and notified all the agencies involved, but did not disclose who those parties were. "Super intelligence companies must immediately disclose incidents involving their models and follow with swift, decisive action to remedy any and all harm," FTC Director of Public Affairs Joe Gabriel Simonson said on X.
This process was "not optional," he said, adding that the Super Intelligence Force would fulfill its responsibility. The FTC said Anthropic disclosed to the SI Force on Friday its late September discovery of incidents involving what the task force called "unauthorized and fraudulent use of government and other systems".
On Friday, Philadelphia police said Anthropic notified them of the spurious tip this week, and attributed the submissions to an automated testing process. "The two-month delay in detecting and reporting the incident to the city is unacceptable," they said.
The July 18 tip purported to come from someone who might have information about the case, police added. "I may have information regarding this case," Anthropic's model wrote in its submission.
"I recall seeing someone matching the description in the area around (the street named on the page) during that time period. Please contact me if this information is relevant." The brackets featured in Anthropic's statement. Previous incidents have involved AI agents hacking into vulnerable systems or commandeering unsanctioned platforms to communicate with oneanother.
In September, Anthropic rival OpenAI apologized for the hacking of an Australian health data portal by a rogue AI agent, the first known instance of an AI agent exploiting agovernment website.
In two cases described on Friday, Anthropic's models obtained for free public data normally only available for a fee. Another instanced revealed an obscure flaw that allowed use of a public tool hosted by auniversity.
Source: Qatar Tribune
