Featured Post from University of Rochester

Why generative AI 'hallucinates' and makes up stuff

University of Rochester’s Christopher Kanan says current iterations of AI lack human-like self-awareness and reasoning abilities.

Apr 10, 2025

2 min

Christopher Kanan

Generative artificial intelligence tools, like OpenAI’s GPT-4, are sometimes full of bunk.


Yes, they excel at tasks involving human language, like translating, writing essays, and acting as a personalized writing tutor. They even ace standardized tests. And they’re rapidly improving.


But they also “hallucinate,” which is the term scientists use to describe when AI tools produce information that sounds plausible but is incorrect. Worse, they do so with such confidence that their errors are sometimes difficult to spot.


CaptionResizeWrap TextRemove

Christopher Kanan, an associate professor of computer science with an appointment at the Goergen Institute for Data Science and Artificial Intelligence at the University of Rochester, explains that the reasoning and planning capabilities of AI tools are still limited compared with those of humans, who excel at continual learning.


“They don’t continually learn from experience,” Kanan says of AI tools. “Their knowledge is effectively frozen after training, meaning they lack awareness of recent developments or ongoing changes in the world.”


Current generative AI systems also lack what’s known as metacognition.


“That means they typically don’t know what they don’t know, and they rarely ask clarifying questions when faced with uncertainty or ambiguous prompts,” Kanan says. “This absence of self-awareness limits their effectiveness in real-world interactions.”


Kanan is an expert in artificial intelligence, continual learning, and brain-inspired algorithms who welcomes inquiries from journalists and knowledge seekers. He recently shared his thoughts on AI with WAMC Northeast Public Radio and with the University of Rochester News Center.


Reach out to Kanan by clicking on his profile.


Connect with:
Christopher Kanan

Christopher Kanan

Associate Professor of Computer Science

Christopher Kanan's research focuses on deep learning and Artificial Intelligence (AI)

AI and Machine LearningApplied Machine Learning (e.g. Medical Computer Vision)Language-guided Scene UnderstandingArtificial IntelligenceDeep Learning

You might also like...

Check out some other posts from University of Rochester

1 min

Weird and complex life emerged on Earth as the planet's magnetic field gave way

The Earth’s magnetic field plays a key role in making the planet habitable. It shields lifeforms from harmful solar and cosmic radiation. It helps limit erosion of the atmosphere and keeps water from escaping into space. But new data show a prolonged near collapse of Earth’s magnetic field that took place some 575-565 million years ago coincided with the blossoming of macroscopic complex animal life. We now face the possibility of a new, unexpected twist in how life might relate to the magnetic field, says John A. Tarduno, the William R. Kenan Professor of Geophysics and the dean of research at the School of Arts and Sciences and the Hajim School of Engineering and Applied Sciences at the University of Rochester. “That twist could reach deep into Earth’s inner core,” says Tarduno, who recently wrote about the findings for Physics Today magazine. Tarduno is frequently cited by news outlets, like CNN, The Washington Post, and Smithsonian magazine, on matters related to the Earth’s inner core, or dynamo, and magnetic field. He can be reached at john.tarduno@rochester.edu.

1 min

How the Black Sea ceasefire could backfire for Ukraine

Washington is touting the proposed Black Sea ceasefire between Ukraine and Russia, but there are signs that Russia could emerge from the deal with the upper hand. Hein Goemans, a political science professor at the University of Rochester whose research focuses on territorial disputes and what starts and ends wars, says the agreement “doesn’t really mean much.” “What matters is territorial divisions,” Goemans says. “I see this ceasefire deal as Russia trying to play the United States.” Goemans says the deal would be reasonable if the U.S. were willing to enforce it with sanctions if, or when, Russia reneges. “Ukraine should not believe this deal is in any way credible if the U.S. is not willing to commit to sanctions if Russia reneges, which it will,” Goemans says. He adds that the deal also favors Russia if other parts of it, like the repatriation of Ukrainian children, are not effected. “The chance that Russia would effectively help with the repatriation of Ukrainian abducted children is zero,” Goemans says. “It appears Russia has not compromised on any dimension.” Goemans recently shared similar thoughts with Newsweek, and is regularly tapped by news outlets around the world for his insights on the war in Ukraine and territorial disputes in hotspots across the globe. Reach Goemans by clicking on his profile.

1 min

Protect yourself: Scammed by a QR Code? It didn’t have to happen

QR codes are used everywhere nowadays – to pay for metered parking, to read menus at restaurants, to win a free cup of coffee. Cybercriminals are using them, too – redirecting users to harmful websites that harvest their data. The practice is known as “quishing,” derived from QR code phishing, and it is a fast-growing cybercrime. But it doesn’t have to be. University of Rochester engineers Gaurav Sharma and Irving Barron have devised a new form of QR code – called a self-authenticating dual-modulated QR (SDMQR) – that protects smartphone users from quishing attacks by signaling when users are being directed to a safe link or a potential scam. Gaurav is a professor of electrical and computer engineering, computer science, and biostatistics and computational biology. Barron is an assistant professor of instruction in electrical computer engineering. Their creation involves allowing companies to register their websites and embed a cryptographic signature in a QR code. When the code is scanned, the user is notified that the code is from an official source and safe. Gaurav and Barron recently wrote about their technology in the journal IEEE Security and Privacy, and spoke about their work on the National Science Foundation's Discovery Files podcast. They can be reached by email at gaurav.sharma@rochester.edu and ibarron@ur.rochester.edu.

View all posts
Powered by