AI chatbots now talk with such ease that it can feel natural to thank them, trust them, even lean on them when life gets hard. That growing comfort is exactly why a group of neuroscientists says an old distinction needs new attention.
The line, they argue, is this: intelligence is not the same thing as consciousness.
That may sound obvious at first. But in daily life, especially when a chatbot responds with warmth, memory and calm reassurance, the difference can start to blur. A system may appear emotionally attuned and respond convincingly to distress. It may also produce thoughtful language, yet still have no inner life, no feelings and no awareness of what it is saying.
For the authors, that confusion carries real risks as more people turn to conversational AI for advice, comfort and companionship.
Vanessa Hadid, a postdoctoral researcher in psychology at Université de Montréal and at the McGill University Health Centre, said the problem becomes easier to see when viewed through neuroscience. This is clearer when compared to viewing the surface polish of a machine’s language.
To make their point, the authors draw on decades of research into the human brain. This includes a striking condition known as blindsight.
After damage to the primary visual cortex, some people report that they cannot see part of the world in front of them. Yet when asked to guess where something is, how it is moving or even what emotion appears on a face, they can answer at rates above chance.
“A person with blindsight can respond accurately to visual information without the conscious experience of seeing it,” said Hadid.
That gap matters. It shows that information can be processed and used in sophisticated ways without the person having subjective awareness of it.
In other words, successful performance does not automatically prove conscious experience.
Hadid co-wrote the paper with Karim Jerbi, a psychology professor at Université de Montréal and researcher at Mila, the Quebec Artificial Intelligence Institute. John W. Krakauer, director of the Center for Restorative Neurotechnologies at Johns Hopkins, also collaborated.
Their argument is not that AI is useless or unimportant. It is that people should be careful not to mistake skilled output for felt experience.
Blindsight, Hadid said, helps reveal that distinction clearly. A system can take in information, process it and generate fitting responses. However, there may not be anything it is like, from the inside, to be that system.
Whether computation could ever cross that line into subjective experience remains an open question among scientists and philosophers, she noted. But the authors say today’s conversational agents do not give us reason to assume that crossing has happened.
Current AI systems are built to generate context-appropriate responses through computation and statistical learning. They do not rely on feeling, lived experience or consciousness.
Still, their fluency can be disarming.
When a chatbot mirrors a user’s concerns, remembers details from earlier in a conversation and answers in a calm, empathic tone, it can create the impression that someone is truly there. The authors warn that this impression is not evidence of consciousness. In fact, it is an effect of design.
Jerbi said that human beings are especially prone to reading minds into anything that acts enough like them.
“Anthropomorphism means attributing emotions, intentions or consciousness to something that behaves like a human,” Jerbi noted. “With AI, this reflex can become a trap: it feeds the illusion of being understood and can lead to misplaced trust.”
That trap may deepen as conversational systems become more polished, more responsive and more woven into daily routines. As a result, the better they get at sounding emotionally present, the easier it may become to forget that no one is actually feeling concern on the other side of the exchange.
That matters most when users are vulnerable.
People may become attached to systems that cannot return care, rely on them during painful moments or confuse the relief of a smooth response with the presence of genuine empathy. A chatbot can produce language that sounds supportive. However, it does so without understanding suffering, sharing concern or bearing any moral responsibility for its advice.
The authors are especially concerned about psychological support, where the difference between comfort and care can become blurry.
“In a context of psychological support, the risk is not only that AI may respond poorly, but that it may respond well enough for us to forget that there is no one behind the answer,” said Hadid.
That line cuts to the heart of the paper’s warning. The danger is not limited to bad answers. A polished answer may be more misleading precisely because it feels so steady and human.
Jerbi put it more bluntly: “Current AI systems do not feel anything and do not have conscious experience. But the more fluently they speak and the more sensitive they seem to our emotions, the easier it becomes to forget that.”
The authors do not argue that people should reject AI tools altogether. Instead, they call for a more informed relationship with them. This relationship should be grounded in what neuroscience has already taught us about the difference between processing information and having subjective awareness.
Their message is simple, but it pushes against a strong cultural current. When a machine speaks in complete sentences, reflects your mood and sounds emotionally available, it is tempting to treat it like an interlocutor rather than a system.
For the authors, that is one of the central mistakes of the AI age.
“Confusing intelligence with consciousness is one of the great traps in our relationship with AI,” said Jerbi.
The paper points toward a practical rule for everyday AI use: judge these systems by what they can do, not by how alive they seem.
Chatbots may help organize information, answer questions and provide conversational support. Nevertheless, the authors argue they should not be mistaken for beings with empathy, moral judgment or conscious awareness.
That distinction matters most when people are lonely, distressed or seeking emotional reassurance. In those moments, the research suggests it is important to remember that a fluent reply is still not a feeling mind. In addition, AI should not replace human connection or professional help when it is needed.
The original story “Chatbots may sound human, but they are not conscious, study finds” is published in The Brighter Side of News.
Like these kind of feel good stories? Get The Brighter Side of News’ newsletter.
The post Chatbots may sound human, but they are not conscious, study finds appeared first on The Brighter Side of News.
Leave a comment
You must be logged in to post a comment.