Interactive AI brings textbooks to life

This novel tool makes tricky physics concepts easier to understand

Spotted: Every child learns best in a different way, and often, reading textbooks full of static diagrams isn’t enough for many kids to grasp new concepts – particularly when it comes to science. To change that, researchers led by Ryo Suzuki at the University of Colorado at Boulder are using AI to help make educational content more interactive and engaging.

The researchers have created a tool called ‘Augmented Physics’, which allows students to turn still diagrams into dynamic, living images using only an iPad, helping them to visualise concepts. Traditional 2D textbooks have been a mainstay of the classroom for decades, but reading about scientific subjects doesn’t necessarily mean a child is going to fully grasp them. With Augmented Physics, students can better understand complex processes – such as momentum, electrical current, cell division, or light refraction – and see them in action.

Students begin by capturing a textbook diagram with an iPad. Then, the technology uses Meta’s Segment Anything model to assign objects in that diagram various roles. For instance, the student could select a skier to ski down a hill. The tool then uses AI to apply physical laws to the diagram, animating the intended motion as it would appear in reality. By adjusting the parameters, students can visualise, in three dimensions, how forces such as gravity, energy, or friction affect the skier’s movement.

Right now, the technology achieves an effective simulation in only 60 per cent of cases, and the team is working on boosting that figure. The researchers also hope to broaden the AI’s capabilities beyond physics, so it could be used to bring documents from a variety of subjects to life using augmented glasses, enabling students to learn in 3D interactive worlds.

Written By: Duncan Whitmore and Matilda Cox