DeepMind Gives AI 'Intuition' by Training It Like a Baby

Babies are cheerful, cuddly, giggling balls of joy. They are also very powerful learning machines. At three months old, they already have an intuition about how things around them behave — without anyone explicitly teaching them the rules of the game.

This ability, dubbed “intuitive physics”, seems very trivial on the surface. If I fill a glass with water and place it on the table, I know that it is an object—something I can wrap in my hand without melting it in the palm of my hand. It will not sink through the table. And if it starts to float, I’ll stare and then immediately run out the door.

Babies quickly develop this ability by absorbing data from their external environment, forming a kind of “common sense” about the dynamics of the physical world. When things don’t move as expected—for example, in a magic trick where objects disappear—they will show surprise.

For AI, it’s a completely different matter. While recent AI models have beaten humans from playing games to solving decades-old scientific puzzles, they are still struggling to develop intuitions about the physical world.

This month, researchers at Google’s DeepMind took inspiration from developmental psychology and built an AI that naturally extracts simple rules about the world through watching videos. Netflix and chill don’t work on their own; AI modelsthere isn’t any learn the rules of our physical world when given a basic idea about objects, what their boundaries are, where they are, and how they move. Similar to babies, AI expresses “surprise” when shown a nonsensical magical situation, such as a ball rolling its way.

Dubbed PLATO (for Learning Physics through Automated Coding and Object Tracking), AI is surprisingly flexible. It only takes a relatively small set of examples to develop its “intuition”. Once it knows that, the software can generalize its predictions about how something moves and interacts with other objects, as well as about scenarios it has never encountered before.

On the one hand, PLATO touches the sweet spot between nature and nurturing. Developmental psychologists have long debated whether learning in infants can be achieved from finding patterns in data from experience alone. PLATO suggests the answer is no, at least not for this particular task. Both innate knowledge and experience are essential to complete the whole learning story.

To be clear, PLATO is not a digital replica of a three-month-old baby—and it was never designed to be. However, it does provide a glimpse into how our own minds can potentially develop.

“Work…pushes the boundaries of everyday experience which can and cannot be explained in terms of intelligence,” comments Drs. Susan Hespos and Apoorva Shivaram, from Northwestern University and Western Sydney University, respectively, were not involved in the study. It may “tell us how to build better computer models that simulate the human mind.”

Common Sense Puzzles

By three months, most babies won’t care if they drop a toy and fall to the ground; they’ve taken the concept of gravity.

How this happens is still confusing, but there are a few ideas. At that age, babies are still struggling to wriggle, crawl, or move. Their input from the outside world is mostly through observation. That’s great news for AI: it means that instead of building robots to physically explore their environment, it’s possible to infuse a sense of physics into AI via video.

This is a theory supported by Dr. Yann LeCun, leading AI expert and chief AI scientist at Meta. In a lecture from 2019, he suggested that babies most likely learn through observation. Their brains construct this data to form conceptual ideas about reality. In contrast, even the most sophisticated deep learning models struggle to establish our sense of the physical world, which limits how much they can engage with the world—making them almost literally think in the cloud.

So how do you measure a baby’s understanding of everyday physics? “Fortunately for us, developmental psychologists have spent decades studying what infants know about the physical world,” wrote lead scientist Dr. Luis Piloto. One particularly powerful test is the expectation violation (VoE) paradigm. Show baby a ball that rolls up a hill, disappears randomly, or suddenly moves in the opposite direction, and baby will stare at the anomaly for longer than he normally would expect. There is something strange.

Space Oddities

In the new study, the team adapted VoE to test AI. They tackled five different physical concepts for building PLATO. Among these are solidity—that is, two objects cannot pass through each other; and continuity—the idea that everything exists and doesn’t flicker even when hidden by other objects (the “peek-a-boo” test).

To build PLATO, the team first started with standard methods in AI with a two-way approach. One component, the perception model, retrieves visual data to parse discrete objects in the image. Next is the dynamics predictor, which uses a neural network to consider the history of the previous object and predict the behavior of the next object. In other words, the model builds a kind of “physics engine” that maps out objects or scenarios and guesses how something will behave in real life. This setting gives PLATO an early idea of ​​the physical properties of objects, such as their position and how fast they move.

Next is training. The team showed PLATO less than 30 hours of synthetic video from an open source data set. This is not a video of a real life event. Instead, imagine an old-school Nintendo-like blocky animation of a ball rolling down a ramp, bouncing off another ball, or suddenly disappearing. PLATO eventually learns to predict how one object will move in the next video frame, and also updates its memory for that object. With training, his predictions on the next “scene” became more accurate.

The team then threw a wrench at the spokes. They gave PLATO a normal and impossible sight, like a ball suddenly disappearing. When measuring the difference between actual events and PLATO’s predictions, the team was able to measure the level of AI “shock”—that goes beyond the roof for magical events.

Learning is generalized to other moving objects. Challenged with a completely different dataset developed by MIT, featuring, among other things, rabbits and bowling pins, PLATO expertly distinguishes between impossible and realistic events. PLATO had never “seen” a rabbit before, but without retraining, it showed surprise when a rabbit defied the laws of physics. Similar to a baby, PLATO is able to capture his physical intuition with just 28 hours of video training.

To Hespos and Shivaram, “These findings also have parallel characteristics that we see in infant studies.”

Digital Intuition

PLATO is not intended as an AI model for infant reasoning. But it does show that harnessing our developing baby’s brain can inspire a computer with a physical sense, even when the software “brain” is literally trapped in a box. It’s not just about building humanoid robots. From prosthetics to self-driving cars, intuitive understanding of the physical world bridges the amorphous digital world of 0’s and 1’s into everyday, run-of-the-mill reality.

This isn’t the first time AI scientists have thought about filling a turbo engine with a little toddler ingenuity. One idea is to give AI a sense of theory of mind—the ability to distinguish itself from others, and the ability to imagine itself in someone else’s shoes. This is an ability that comes naturally to children around the age of four, and if incorporated into AI models, can dramatically help them understand social interactions.

This new study builds on the early months of our lives as a rich resource for developing AI with common sense. For now, this field is still in its infancy. The authors release their dataset for others to build on and explore the ability of AI models to interact with more complex physical concepts, including videos from the real world. For now, “this research can serve as a synergistic opportunity across AI and developmental science,” said Hespos and Shivaram.

Image Credit: thedanw from Pixabay

#DeepMind #Intuition #Training #Baby