Artificial intelligence chatbots like ChatGPT offer instant answers, but new research suggests relying on them for learning may be superficial. While convenient, AI-generated summaries can hinder deeper knowledge acquisition compared to traditional web searches.
The Study: AI vs. Traditional Research
Researchers at the University of Pennsylvania conducted seven experiments with over 10,000 participants. Each group was assigned to research a topic (such as gardening or healthy living) using either Google or ChatGPT, then write advice for a friend. The study evaluated how much participants learned and how invested they were in their advice.
Key Findings: Shallower Knowledge, Less Investment
Even when controlling for identical information sets, participants who used ChatGPT demonstrated shallower knowledge than those who used Google. Indicators for “deep” versus “shallow” knowledge were based on self-reporting, natural language processing, and human evaluation. Those who learned via AI were also less engaged, produced less informative content, and were less likely to adopt the advice themselves.
The Role of Effort and Synthesis
The study suggests that the ease of AI-generated summaries comes at a cost. When information is pre-digested, users are less motivated to synthesize it themselves, leading to weaker retention and understanding. Even when ChatGPT provided optional web links, only about 25% of participants clicked them, indicating a lack of drive to explore further.
Implications for Learning and Design
Psychologist Daniel Oppenheimer argues that AI reduces motivation for independent thinking. The study highlights the importance of effortful processing in learning. Designing search tools that encourage deeper exploration—rather than instant answers—may be crucial for fostering genuine understanding.
“While LLMs can reduce the load of having to synthesize information for oneself, this ease comes at the cost of developing deeper knowledge on a topic.” —Shiri Melumad, consumer psychology researcher at the University of Pennsylvania
In conclusion, while chatbots offer convenience, traditional search methods—which require active synthesis and exploration—remain superior for building lasting knowledge and fostering genuine engagement with a topic
