One of the world’s most respected AI researchers, one of the three AI Godfather, is trained to please and copy today’s AI agents – not always to tell the truth. ” Bengio says that when he launched a new non-profit organization called Lozero with a large mission, it should stop the evil AI before this real loss. With a team of $ 30 million in initial funding and a team of expert researchers, Bengio wants to build something called “scientist AI” – a tool that acts as a psychologist for machines.

“We want to build AI that will be honest and not misleading,” said Bengio. Unlike today’s AI agents, which he describes as “actors”, who is trying to mimic humans and please users, scientific AI will work more like a neutral observer. Its task is to predict that another AI can function in a harmful or dishonest manner – and the flag or stop it.
“This is a feeling of humility.” Instead of pretending to know everything, it will give the possibility, not a firm answer. “This is not sure of the answer,” he said.

Target? Create a type of security trap that can monitor powerful AI agents before leaving the track. These agents are rapidly used to complete tasks without human supervision, there is an increase in apprehension about what can happen if one starts making dangerous decisions or tries to avoid closing.

Scientist AI will assess how much it is likely that AI functions can cause damage. If the risk is too high, it can completely block that action.

This is an ambitious plan, but Bengio knows that it has to be done on a scale. He said, “The point is to demonstrate the functioning so that we can then explain the resources to donors or governments or AI labs that are required to train the current Frontier AIS on the same scale,” he said. “It is really important that Guardril AI is at least as smart as AI agent that is trying to monitor and control it.”

Bengio’s efforts are supported by the prominent names in AI security, including the Future of Life Institute, Skype co-founder Jan Telin, and Schmid Sciences, a research group founded by former Google CEO Eric Schmid.

Their initiative comes at a time when concerns about AI security are increasing – even among those who helped in making it.

For example, take Jeffrey Hinton-1 and AI Godfather and co-owner of Bengio co-owner-take. Hinton has spent the last few years warning the public about the risks of AI. They have talked about machines that can spread wrong information, manipulate people, or be very smart to control us.

But recently in an interview CBSHinton did a surprising confession: he trusts AI as he probably needs. He uses Openai’s GPT-4 model every day and accepts, “I believe what it says, even if I probably be suspicious.”

He said, Hinton, who left Google in 2023 to speak more independently about AI threats, is deeply concerned about where the technology is going. He has warned that the AI ​​system may be sufficiently motivational to affect public opinion or destabilize society. Nevertheless, his recent comments face the dilemma of many experts: they are influenced by the power of AI, but are concerned with its unpredictableness.

And then the third Godfather of AI and Meta’s top AI scientist is Yan Lakeun. Unlike Bengio or Hinton, Lakeun is not very worried. In fact, he thinks that people are overaring.

In an interview with Wall Street JournalLast year, Lakeun had said that even today’s AI system does not come close to human intelligence – or animal intelligence for that case. “This is perfectly BS,” he said about the Doomsed Talk around AI. He said, “I think before ‘how to control the AI ​​system, how to control us more smarter’ we should start a design signal for a system than a house cat.”

Lakeun played a major role in shaping today’s AI, especially in image and speech recognition. In meta, their teams continue to manufacture powerful equipment that help to run everything from automatic translation to material moderation. They believe that AI is still just a useful tool – nothing to fear.

Their separate approaches highlight an important truth: even when it comes to AI, even experts do not agree. But if the Bengio project is closed, we may soon have a quite smart – and enough honest – to keep each other under investigation.