AI godfather Bengio warns not to give power to AI or we may lose the power to shut it down
AI godfather Yoshua Bengio has expressed concern over the debate over empowering artificial intelligence (AI). According to the Canadian computer scientist, this could leave humans powerless if things go wrong in the future.
Artificial intelligence (AI) models are improving rapidly. AI models today are much faster and productive than they were a year ago. This has sparked debate over whether AI systems should be given legal rights. In short, this means that AI systems can be treated like humans with the same rights. However, AI godfather Yoshua Bengio believes this science-fiction scenario could spell disaster for humanity.
Bengio, who is considered one of the three AI godfathers along with Geoffrey Hinton and Yann LeCun, strongly opposes giving rights to AI. In an interview with The Guardian, he indicated that this would amount to nothing less than ‘granting citizenship to hostile extraterrestrials’. Yoshua Bengio made it clear that people need to reconsider this demand. “It would be a big mistake for people to demand that AI have rights,” he said.
Humans could fail to shut down AI if given legal powers
Now, on the surface, giving AI legal rights might not seem like a drastic idea. After all, humans are increasingly using AI not only for information, but also as a way to receive emotional support. However, Yoshua Bengio pointed out that humanity may lose control over AI in the long run. He explained, “Frontier AI models already show signs of self-preservation in experimental settings today, and ultimately empowering them would mean we’re not allowed to turn them off.”
Instead, AI godfathers recognized the need to build guardrails as AI models become more advanced. If these models get out of hand this will allow humans to eventually stop the AI. “As their capabilities and degree of agency grow, we need to make sure we can rely on technical and social guardrails to control them, including the ability to shut them down if needed,” Bengio said.
But how did this debate start?
As the popularity of AI chatbots has grown, concerns regarding ‘AI welfare’ have increased. In August 2025, Anthropic said it would let its Cloud Opus 4 models turn off potentially “annoying” interactions with users. The firm claimed that this was done to protect ‘AI welfare’. Tech billionaire Elon Musk, who leads XAI, chimed in on the conversation at the time, writing, “It’s not okay to torture AI.”
However, for Yoshua Bengio, the belief that chatbots could be considered conscious was leading to ‘bad decisions.’ The AI godfather reiterated that this could be like giving citizenship to aliens who have bad intentions. Bengio said, “Imagine that some alien species came to the planet and at some point we realized that they had nefarious intentions for us. Do we give them citizenship and rights, or do we protect our lives?”