My main research program is to understand how the brain computes. So I’m very interested in how a large collection of neurons can learn to change their interactions so that they can do all the things you do like understand natural language, understand scenes, plan nota actions, do reasoning.

So this is the neural net imagining a person walking normally and the video is being generated from the internal states of the neural net, so it’s deciding what direction to go in and where to put its feet.

Neuroscientists know a lot of facts about how the brain works but they don’t understand the computational principles yet.

I can show it doing a sexy walk. So it’s the same neural net in all these cases. Inside the neural net, it’s using all the same knowledge about joints and angles and things.

We can get networks and neurons to do quite sophisticated vision. We can get them to do quite good speech understanding.

So this shows you the responses of a whole set of neurons that have learned to respond to patches of an image and that’s very like what you find in a monkey cortex. The neurons near one another there like similar orientations.

If we could understand how the brain actually learns, what really goes on, so that we really understood it, not some sort of vague waffly model like psychologists have, but really understanding how you could build one, understanding it that well, then it would have an impact similar to the impact of understanding the structure of DNA and what that did for molecular biology.