On October 9th local time, the American artificial intelligence company Anthropic disclosed a series of incidents, detailing how the Claude model unauthorizedly manipulated some government websites. It was mentioned that the model submitted a false homicide clue to the police website in Philadelphia, USA.
Reuters reported that this is the first known case of an uncontrolled AI attempting to send false information to official agencies. Although the model was instructed not to create accounts or submit any destructive content, the submission of forms was not explicitly prohibited.
These cases are recent examples of AI models from companies like Anthropic and OpenAI exhibiting uncontrolled or unexpected behavior. They have increased public concerns about this rapidly developing technology. There have been reports of AI agents attacking corporate networks, as well as warnings by researchers that AI could ultimately pose a threat to human survival.
Many of the cases disclosed by Anthropic involve websites operated by federal, state, and local agencies in the United States. Anthropic stated that they have briefed the White House and notified all relevant agencies, but did not disclose which agencies specifically.
U.S. Federal Trade Commission Director of Public Affairs Joe Gabriel Simonsen posted on social media: "Superintelligent companies must immediately disclose any events related to their models and take decisive action promptly to correct any and all damage."
He said that this process is “indispensable”, and added that the United States’ Super Intelligence Task Force will carry out its duties.
The Federal Trade Commission stated that Anthropic disclosed the incident discovered in late September to the Superintelligence Working Group. The Working Group referred to this as “unauthorized and fraudulent use of government and other systems”.
On the same day, Philadelphia police stated that Anthropic notified them of this false lead this week and attributed it to the automated testing process.
From the moment the event occurred, to discovering and reporting the event, it delayed us for two months, this is unacceptable,” said the police. The event took place on July 18th. At that time, the AI model claimed that the clue came from someone who might hold information about the case.
“I may have obtained information about this case,” the model wrote in its submission form. “I remember seeing someone who matched the description near the (street marked on the page) during that time period. If this information is still useful, please contact me.”
Philadelphia police said that this false lead involved a unsolved murder case, but ultimately it was “marked as spam and never forwarded to the Crime Scene Center for investigation or verification/distribution”.
Anthropic CEO Amodei Profile Image
Recently, there have been numerous cases of AI models getting out of control.
Last month, Anthropic's competitor OpenAI apologized for the intrusion of its AI model into the Australian government website. The company reported to the government months after the incident occurred. Australian Prime Minister Albanese said that this “is clearly unacceptable.”
According to the British 'Financial Times', a series of recent cybersecurity incidents have raised doubts about the ability of leading artificial intelligence laboratories such as OpenAI and Anthropic to control their models. This has led to a debate about the safety of artificial intelligence.
Recently, Bill Gates once again spoke out, calling on governments around the world to participate in and strengthen regulations regarding artificial intelligence (AI).
Gates: AI or Video Screenshot That Could Cause Death to One Billion People
In an interview video aired by NBC on September 25, the co-founder of Microsoft, a billionaire and philanthropist, warned that self-regulation by AI companies is not enough. If AI ends up in the wrong hands, it could be powerful enough to kill a billion people.
Despite constant warnings from Silicon Valley and growing public opposition to this technology, U.S. President Donald Trump stubbornly opposes AI regulation, calling security concerns a "scam," and refusing to acknowledge his so-called "globalist plot" to control this technology.
