Students using a chatbot designed to ask questions learn more than those using ChatGPT, but still don't solve problems better

Researchers compared how secondary school students use custom chatbots designed to ask guiding questions versus general-purpose chatbots like ChatGPT for science problems. Students asked more questions and thought harder with the custom version, but both groups solved the actual problems equally well — suggesting that better thinking during problem-solving doesn't automatically produce better answers.

This is the uncomfortable finding in AI education: a tool can change how students think without changing what they produce. Teachers have assumed that if you get students to engage more deeply with material, they'll perform better. This study suggests that's not automatic. The custom chatbot design worked — students did think harder and relied less on copying answers. But that cognitive engagement didn't translate to solving problems faster or better. What matters now is whether schools will invest in custom pedagogical chatbots if the payoff isn't measurable performance gains, or whether the thinking process itself becomes the metric that counts.