Why generative AI 'hallucinates' and makes up stuff

University of Rochester’s Christopher Kanan says current iterations of AI lack human-like self-awareness and reasoning abilities.

Apr 10, 2025

2 min

Christopher Kanan

Generative artificial intelligence tools, like OpenAI’s GPT-4, are sometimes full of bunk.


Yes, they excel at tasks involving human language, like translating, writing essays, and acting as a personalized writing tutor. They even ace standardized tests. And they’re rapidly improving.


But they also “hallucinate,” which is the term scientists use to describe when AI tools produce information that sounds plausible but is incorrect. Worse, they do so with such confidence that their errors are sometimes difficult to spot.


Christopher Kanan, an associate professor of computer science with an appointment at the Goergen Institute for Data Science and Artificial Intelligence at the University of Rochester, explains that the reasoning and planning capabilities of AI tools are still limited compared with those of humans, who excel at continual learning.


“They don’t continually learn from experience,” Kanan says of AI tools. “Their knowledge is effectively frozen after training, meaning they lack awareness of recent developments or ongoing changes in the world.”


Current generative AI systems also lack what’s known as metacognition.


“That means they typically don’t know what they don’t know, and they rarely ask clarifying questions when faced with uncertainty or ambiguous prompts,” Kanan says. “This absence of self-awareness limits their effectiveness in real-world interactions.”


Kanan is an expert in artificial intelligence, continual learning, and brain-inspired algorithms who welcomes inquiries from journalists and knowledge seekers. He recently shared his thoughts on AI with WAMC Northeast Public Radio and with the University of Rochester News Center.


Reach out to Kanan by clicking on his profile.


Connect with:
Christopher Kanan

Christopher Kanan

Associate Professor of Computer Science

Christopher Kanan's research focuses on deep learning and Artificial Intelligence (AI)

AI and Machine LearningApplied Machine Learning (e.g. Medical Computer Vision)Language-guided Scene UnderstandingArtificial IntelligenceDeep Learning
Powered by

You might also like...

Check out some other posts from University of Rochester

Can Music Legends Rewrite Their Legacy? featured image

2 min

Can Music Legends Rewrite Their Legacy?

The Stones didn’t need another hit. With six decades of chart-topping albums, sold-out tours, and songs woven into popular culture, their place in rock history has long been secure. Yet the band’s scheduled release of another studio album, “Foreign Tongues,” on July 10, raises questions about how late-stage work can impact the legacy of the Stones and other enduring musical acts. For John Covach, director of the Institute of Popular Music at the Univeristy of Rochester and a leading scholar of rock music, that’s where the real story is.  “Every late-career album asks us two questions,” Covach says. “What does it say about where the artist is now? And does it change how we hear everything that came before?” It’s a question that could be applied to artists from Bob Dylan and Paul McCartney to Bruce Springsteen and Neil Young. Sometimes late work reflects an unexpected creative renaissance. Sometimes it simply reinforces an artist’s legacy. Sometimes it challenges audiences to rethink musicians they thought they already understood. Sometimes it becomes a footnote to their career. “An artist's latest act can in many ways be as revealing as their first,” Covach says. Covach, who co-edited The Cambridge Companion to the Rolling Stones (Cambridge University Press 2019) and whose online course on the music of the Rolling Stones has enrolled thousands of students worldwide, says reporters covering the Stones’ new album have an opportunity to explore broader issues that resonate across popular culture: • Can new work meaningfully change an artist’s historical legacy? • Why do some musicians continue creating well into their seventies and eighties while others stop? • Can a new release introduce younger listeners to artists whose biggest hits predate them by decades? • How do critics — and fans — judge new music from legendary performers differently than music by younger artists? • What determines whether late-career work becomes an essential part of an artist's catalog — or a historical footnote? Covach has spent decades studying the evolution of popular music, and his books and scholarship have helped shape how the genre is taught. He is also a frequent media commentator on the cultural significance of major artists and musical milestones. Click on his profile to connect with him.

Research Matters: Water, Water Everywhere — and Lots to Drink featured image

2 min

Research Matters: Water, Water Everywhere — and Lots to Drink

Researchers at the University of Rochester have discovered a better way to turn seawater into drinking water as climate change, population growth, and drought intensify pressure on freshwater supplies. Desalination, as the process of converting saltwater to freshwater is known, has been used for some time. But desalination methods commonly used today have significant drawbacks: they require large amounts of energy and generate brine waste that can damage marine ecosystems. Enter University of Rochester optics and physics professor Chunlei Guo and his research team, who have developed a solar-thermal desalination technology that converts seawater into drinking water without chemical additives and without producing the harmful brine. Their system uses a specially engineered solar panel made of “superwicking” black metal etched with ultrafast lasers that allow it to absorb light and attract water. The panels have a laser-treated “active” region that pulls a think layer of water across the surface, absorbs sunlight, distills the water, and deposits leftover salts and minerals onto the untreated “passive” region. The technology also transforms waste into a resource. Instead of generating brine, the process captures salts in solid form, creating opportunities to recover valuable minerals. Guo's team has already demonstrated the ability to extract lithium, a critical component in rechargeable batteries, from salt-rich water sources. For reporters covering sustainability innovation, Guo is available to discuss: • Why desalination is becoming increasingly important worldwide • The environmental challenges associated with current desalination technologies • How solar-powered desalination works • The role of advanced materials and laser engineering in water purification • Recovering valuable minerals such as lithium from seawater • The future of sustainable water and resource management With an estimated 2.2 billion people worldwide lacking access to safely managed drinking water, Guo's research offers a glimpse of how next-generation technologies could help address both global water shortages and growing demand for critical minerals. Researchers recently explained their method in a paper published in Light: Science & Applications. Journalists can connect with Guo by contacting Luke Auburn, director of communications at the University of Rochester’s Hajim School of Engineering and Applied Sciences, at luke.auburn@rochester.edu.

Spielberg's "Disclosure Day" Revives the UFO Debate. But What Would Real 'Disclosure' Mean? featured image

2 min

Spielberg's "Disclosure Day" Revives the UFO Debate. But What Would Real 'Disclosure' Mean?

What if the government finally revealed the truth about UFOs and extraterrestrial visitors? That’s the premise of the new Steven Spielberg film “Disclosure Day,” which the director has said was inspired by the U.S. government’s release of previously classified records related to unidentified aerial phenomena (UAPs) that sparked congressional hearings and renewed interest in so-called “disclosure.” But to University of Rochester astrophysicist Adam Frank the real question isn't whether the government is hiding secrets. It's what would count as actual evidence of extraterrestrial interaction. “Over the last several years, we’ve had hearings, testimony, and lots of extraordinary claims,” Frank says. “What we haven’t had is the one thing science requires: hard physical evidence.” Frank, an award-winning science communicator, astrophysicist, and leading expert on the search for extraterrestrial life, says the distinction matters. Stories, rumors, and secondhand accounts may generate headlines, but they don't constitute proof. "What true disclosure would mean is simple," Frank says. "It wouldn’t be stories about alien spaceships, but the actual spaceships. Not stories about alien bodies, but actual physical evidence that independent scientists around the world could examine and verify." As media coverage surrounding UFOs, government transparency, and extraterrestrial life intensifies, Frank offers a grounded scientific perspective on what we know, what we don't know, and how science separates possibility from proof. Frank is available to discuss: • The science behind UFO and UAP investigations • What constitutes evidence of extraterrestrial life • Why government disclosures have so far failed to provide proof • The search for life elsewhere in the universe • How Hollywood portrays alien contact versus scientific reality • Why scientists remain open to — but skeptical of — extraordinary claims "The universe is vast, and the possibility of life elsewhere is real," Frank says. "But if we're going to claim aliens have visited Earth, then we need evidence that meets the same standards we would demand for any other scientific discovery." Frank is a frequent on-air commentator for live interviews and segments in national media outlets and the author of The Little Book of Aliens (Harper Collins, 2023). He also regularly contributes to written publications, including Forbes, The Atlantic, The New York Times, and Scientific American. He is a recipient of the Carl Sagan Medal, which recognizes and honors outstanding communication by an active planetary scientist to the general public. Click on Frank's profile to connect with him. 

View all posts