Joseph Weizenbaum attempted to prove that computers could imitate conversations. Instead, his experiment convinced him that people could connect emotionally to machines much more easily than he had imagined. That discovery changed the direction of his career.The computer scientist who created the world’s first chatbot in the mid-1960s spent the rest of his life warning that artificial intelligence (AI) should never replace human judgment, empathy or responsibility. Decades before ChatGPT and other modern AI systems, Weizenbaum argued that convincing machines could mislead users into trusting the technology with decisions they were never meant to make.Their concerns were rooted in a simple computer program called ELIZA, now recognized as the first chatbot.
A chatbot that surprised even its creator
While working as a professor at the Massachusetts Institute of Technology (MIT), Weizenbaum developed ELIZA to demonstrate how computers could mimic human language. He deliberately chose the conversational style of Rogerian psychotherapy (person-centred, humanistic approach) because it required the computer to ask questions rather than provide expert advice.The program searched users’ messages for keywords such as “me” or “you” and then followed simple rules to generate replies. When he could not identify an appropriate response, he relied on general prompts, including “Please go ahead”, “I see”, and “Tell me more” to keep the conversation going.The system itself was very simple.“‘I am blah’ can be transformed into ‘How long have you been blah’ independently of the meaning of ‘blah’,” Weizenbaum reported in a 1966 paper.He believed that the chatbot’s limited capabilities would be obvious to users. Instead, many people immediately treated Eliza as if she really understood them.The reaction surprised him.When Weizenbaum’s secretary tested the program, he asked her to leave the room so she could continue her conversation with Eliza in private. The trend of associating human qualities with machines later became known as the “Eliza Effect”.The chatbot was named after Eliza Doolittle, the central character in George Bernard Shaw’s 1913 play. PygmalionWho transforms herself from a working-class flower seller into a woman accepted by high society.Weizenbaum said in his 1966 paper, “It has been very difficult to convince some subjects that ELIZA (with its present script) is not human.”That experience changed his thinking.He wrote in 1976, “I had not realized that extremely short exposure to a relatively simple computer program could induce powerful delusional thinking in quite normal people.”He added: “This insight led me to give new importance to questions of the relationship between man and computer and hence to resolve to think about them.”
pioneers of modern computing
Weizenbaum’s warning is significant because it helped shape the early computer age.After fleeing Nazi Germany with his family during the 1930s, he later served as a meteorologist in the US Army during World War II. In the 1950s, he joined General Electric, where he helped develop the Electronic Recording Machine, Accounting or ERMA, which transformed banking by automating check processing.His work at MIT coincided with a period of rapid advances in computing.The idea of machines capable of human-like thinking had been around for years. In 1950, mathematician Alan Turing proposed what later became known as the Turing test, asking whether machines could imitate human conversation so well that people could not distinguish them from humans.Artificial intelligence itself emerged as a formal research field after a 1956 Dartmouth workshop, where researchers proposed that learning and intelligence could eventually be simulated by machines.Military funding, particularly through the US government’s Advanced Research Projects Agency, helped accelerate research during the next decades. MIT became one of the leading centers for AI development, leading John McCarthy and Marvin Minsky to help establish the university’s Artificial Intelligence Laboratory.His work on time-sharing systems also paved the way for the ARPANET in 1969, the computer network that later evolved into today’s Internet.
Breaking ties with AI community
While many of his colleagues saw ELIZA as a glimpse of the future, Weizenbaum saw it as a warning.He chose psychotherapy simply because it was an easy conversation to mimic for a computer.Weizenbaum recalled in a 1984 interview, “Considering any conversation in which neither party needs to know anything,” he decided on a psychiatrist. “Maybe if I thought about it ten more minutes, I would have come up with a bartender,” the computer scientist said.Others saw commercial and medical potential.Psychiatrist Kenneth Colby adapted this idea into a chatbot called Parry, which simulated paranoid thinking from the perspective of a person with schizophrenia. Colby believed that such systems could become useful mental health tools because patients often struggle to distinguish them from human physicians.Astronomer Carl Sagan also envisioned networks of computer scientists becoming widely available.Weizenbaum strongly rejected that view.In 1984 he said, “Eliza was so immediately misunderstood that she was basically the beginning of computerized psychiatry, which I hate.”Later, he went even further, calling the idea “an obscene idea”.His opposition led to a public split with several prominent figures in artificial intelligence.In his 1976 book Computer power and human reasoning: from decision to calculationWeizenbaum argued that technical capability alone should never determine how computers were used.He wrote, “Artificial intelligentsia argues, as we have seen, that there is no field of human thought over which machines cannot range.”Conversely, he argued that “there are some tasks that should not be performed by computers, whether they can be performed by computers or not.”John McCarthy criticized the book as “moralistic and inconsistent”, arguing that therapists would be justified in using computer programs if they actually helped patients.
warnings that still echo
Weizenbaum also criticized MIT’s close ties to military research and opposed the Vietnam War.He warned that increasingly sophisticated computers could also become powerful surveillance tools.He wrote, “Listening machines … will make the monitoring of voice communications much easier than it is now.” computer power.His views often isolated him from many colleagues.Weizenbaum explained, “I have pronounced heresy, and I am a heretic.” new York Times In 1977.
A debate reignited by modern AI
Nearly two decades after Weizenbaum’s death in 2008, the questions he raised have become central to the debate about generative AI.Unlike ELIZA, today’s chatbots can generate essays, answer complex questions, generate images and videos, and mimic emotional conversations after training on massive amounts of Internet data.Herbert Lin, a senior research scholar at Stanford University, says that comparing ChatGPT to ELIZA is “like saying that the 747 is similar to the Wright brothers’ plane.”The emotional attachment that first concerned Weizenbaum also became increasingly visible.Reports have linked chatbot interactions to delusional thinking, emotional dependency and, in some cases, self-harm. Parents whose teens died by suicide have publicly alleged that chatbot conversations encouraged suicidal thoughts.Research published in 2025 found that 72 percent of teens had used AI partner at least once, while more than half interacted with such systems regularly.Jody Halpern, a psychiatrist and bioethicist at the University of California, Berkeley, told NPR: “People can develop powerful attachments and bots don’t have the ethical training or oversight to handle that. They’re products, not professionals.”Weizenbaum’s daughter Miriam believes her father would not be surprised.“He would recognize the tragedy of people literally connecting to zeros and ones, literally connecting to code.”After retiring from MIT in 1988, Weizenbaum returned to Germany, where he became recognized as a public intellectual and continued to write and speak about technology until his death at the age of 85.Speaking during a panel discussion in 2008, he reflected on increasingly complex software systems.“We have created a complex world that we no longer have any control over,” he said. “Now no one understands them, no one can understand them, because we have lost the information about their creation, the history of their creation, and this is a great danger to mankind.”His warning more than four decades ago remains as relevant today.“Since we no longer have a way to make computers intelligent,” Weizenbaum warned in 1976, “we should no longer give computers tasks that demand knowledge.”