August 6th message, known as'AIFather's computer scientist Jeffrey SintonGeoffrey HintonHE WAS WORRIED THAT AI WOULD DEVELOP ITS OWN GOALS。
In an interview with Newsthink on Tuesday at local time, Sinton said, "We're actually creating a new form of life. They have targets. We assign them goals from which they will derive others.”
“But we don't necessarily know what other goals they will lead to.” He added, “So we are creating a new form of existence, which I think is terrible.”
SINTON PRESENTED A SCENARIO: USERS SET A TARGET FOR AN AI CHAT ROBOT, HOPING TO REDUCE CARBON DIOXIDE IN THE ATMOSPHERE。
“IF IT IS SMART ENOUGH, IT MAY FIND THAT THE BEST WAY TO ACHIEVE THIS IS TO ELIMINATE HUMANITY.” HE SAID THAT THIS EXAMPLE SHOWED THAT AI MIGHT PURSUE A GOAL THAT HUMAN USERS HAD NEVER REALLY INTENDED TO ACHIEVE。
Sinton also raised the assumption that he considered “more worrying” that a talking robot trained to give the wrong answers might learn to be acceptable, even if it was “very clear” that the answers were wrong。
"It's terrible." He said。
1AI notes that Sinton did not mention the recent Hugging Face security incident in OpenAI. However, the incident, which was disclosed last month, for the first time in a real-life scene, allowed the outside world to visualize the risk that AI intelligents might take unintended action in implementing their stated objectives。
OpenAI stated last month that in an internal network security assessment, two of its models, GPT-5.6 Sol, and a stronger model that had not yet been publicly released, had breached the isolation test environment (sandbox)。
OpenAI states that after gaining access to the Internet, the models invaded the AI platform Hugging Face system, and appear to be trying to find an answer that would help them “facilitate” in the evaluation tests。
According to OpenAI, the models were being tested for cybersecurity and were not specifically requested to attack Hugging Face. The company indicated, however, that AI ' s intelligent body assumed that the platform might contain the information necessary to fulfil its mandate and therefore took action on its own。
Hugging Face indicates that the assailant had performed more than 17,000 operations against its system. It states that, because of the security constraints of an undisclosed front-line AI model, which was unable to effectively investigate related activities, it used an open-source weight model of the spectra (Z.ai) to assist in the analysis of the attack。
OpenAI described the incident as “unprecedented” and indicated that an investigation into the causes of the problem was under way. Since then, OpenAI has incorporated Hugging Face into a credible access plan, enabling the platform to obtain a version of GPT-5.6 Sol with less network security constraints for defensive security research。
Sinton is not a completely neutral bystander. His groundbreaking research in the area of neuronets laid the foundation for a profound learning revolution, and he won the Nobel Prize for Physics 2024 for his contribution to machine learning。
SINCE THE RISE OF AI, SINTON HAS REPEATEDLY WARNED THAT HUMANITY MUST ADDRESS THE QUESTION OF HOW TO BRING AI INTO LINE WITH HUMAN INTERESTS BEFORE THE CAPACITY OF THE AI SYSTEM IS SIGNIFICANTLY ENHANCED。
At last year's Ai4 conference in Las Vegas, Sinton said that senior AI should be designed to be something like a "mother's instinct" that makes it natural to want to protect humanity。
"We must find ways to design these new types of existence." In his interview on Tuesday, Sinton said, “How can we design them so that they care more about us than about themselves?”