World Labs' Fei-Fei Li on Creating Large World Models

- Everyone is focused on LLMs, ChatGPT, Claude, large language models, but you have raised a billion dollars to build something different. large world models. Make the case for us. what is the bet? you are making. that others aren't. Right, so this is my co-founded startup, World Labs, and we are or in spatial intelligence. and the means to spatial intelligence is building a large world model. So what is the case for us? The case for us is a 500 million year story. is that Animal intelligence starts with seeing and moving. in the physical world that evolution began with us as animals.

knowing what the world is, knowing who we are, knowing how to move around it, interact with it, and much of life, human life, human work life, human private life. has a lot to do with perceiving, understanding, reasoning, interaction, with the world. including imaginary world of creativity, of productivity as virtual worlds. So unlocking that capability in machines unlocking the capability of generating alien 3D, 4D worlds, unlocking the capability of reasoning within... any world. unlocking the capability of teaching agents or robots or assisting humans to interact with the world, is what spatial intelligence is about.

And that's what we are focusing on. - So what can world models do ultimately that LLMs will never be able to? Can words put down fires? - Can words cook an omelet? I think there's so much, right? So we, for example, creativity, people design, people whether we're designing interior space, we're designing machines, we're designing homes, we're designing stories. So much of that is beyond words. We also use agents. whether we use agents in virtual world, whether it's for entertainment like gaming, or for a more serious industrial applications whether it's the digital twin design or inspection or optimal or what kind of or many kind of optimization tasks.

or we build robots to help us to do a lot of things. from putting down fire to helping healthcare scenarios to manufacturing All those are application downstream applications of unlocking spatial intelligence and building world models. So what do you think the chat GPT moment for world models will be? Like, how will we know this has arrived? - Yeah, that's a great question, Emily, because Chat is such a consumer behavior that chat, GPT moment, tends to be used to describe a viral public consumer moment of getting so close to what AI can do.

In the world of world models, Um, the kind of spatial intelligence we're trying to unlock, um, I'm still trying to figure out if there is a corresponding consumer moment, because the kind of applications we are talking about, tends to be first going to the professionals, professional creators, professional designers, professional developers, professional researchers, engineers who use it for robotics and industrial design and all that. Maybe we will not necessarily have a consumer moment. But maybe we will. And you know, I would love to design my home in a much easier way and just change the color of the curtain, you know, with a click.

All right, that sounds pretty cool. So in the last six months, Jan LeCun left Meta to work on world models, Google shipped Project Genie, NVIDIA has its own world models. Cosmos, NVIDIA is also one of your investors. What do you have that they don't, and which competitors out there worry you the most. YANG YANG: Yeah, so first of all, we started World Labs in 2024. I still remember when we were out talking about world models and spatial intelligence It was just a year after TASTPT, people were still totally talking about LLM.

So we we really had a head start and understanding that this is going to be the next frontier of AI. I'm very excited by that. So what do they have? We don't want first of all, I think we have an incredible team. We have the conviction. They don't have the godmother, that's for sure. the world is big and I think this is just like LLMs. I think there will be many companies doing incredible work in world models. Just as 24 hours ago, I, we kind of got fed up that the word world model has been so confusing and being used in so many different ways that we actually put out a blog just explaining what a functional taxonomy of world model is instead of moving everything together.

And the way I see it is right now there are three ways. of calling world models when it comes to spatial intelligence, One is what I call a renderer, when the model puts beautiful pixels on the screen. mostly like video generation model. And the consumer is mostly human eyeballs. And while the model commits to beautiful pixels on the screen, it doesn't necessarily commit to physics and dynamics and geometric correctness, because that's for just uh... consuming, human eyeball consuming, not necessarily for computation and other tasks. Then another kind of world model is what we call a planner.

That is more for machines, more for robots, where it outputs whatever the input is, the state of the world or the action, it outputs a correct action. to take to the next step. And you see that kind of world model a lot for robotics applications, and you hear that in that context. The third kind, which I think is the linchpin, of the three is a simulator, is that it actually is consumed by humans as well as machines is trying to respect the structure, the physics, and the dynamics of the world, and really simulate the 3D and 4D information of the world, as well as the semantic information.

And a simulator could become a renderer, the simulator could become a planner, but this layer is a huge critical path in my opinion, to unlock spatial intelligence. And that's what WorldLab is working on. All of this rolls up into robotics, so I want to get your take on the field, and humanoids in particular. Funding for humanoids hit $6 billion, but you know. They still can't load my dishwasher as fast as I can. They still can't go get my Amazon packages. Will world... models, world labs, close the gap between hype, In reality, That's a loaded question, Emily.

First of all, that is my job. Yeah, I get it. First of all, robotics is going to be one of the most important revolution in human industrialization. Six billion dollars is too small. Right. If you look at self-driving cars investment, if you look at language models investment, it took way more than six billion dollars. I'm not saying We now I think it will take time. to invest. and it will also hopefully not take the hype but take the thoughtfulness to invest in the right effort. And for example, unlocking world modeling and spatial intelligence and simulation layer, all this is part of that, important effort.

Well, are we going to close the gap? I do believe World Labs is working on one of the most critical technology in the spatial physical intelligence. And obviously that's the hope. Mm-hmm. You've been more measured on AI safety, skeptical of the doom narrative, but also of heavy handed. regulation. When you look across the industry, Where do you seal real safety work versus safety... theater. Is anyone getting it right? So in general, I've been just more measured on every Every rhetoric makes me very boring, to be honest. I think there's just so much hype.

There is so much hype. Obviously, we need to build the right technology. We need to guardrail the technology. Whether you use the word responsible, you use the word safety, you use the word trustworthy, building the right technology and product so that it can empower, enhance, augment humanity, not harm them. is the goal of any any work we do, whether it's AI or not. where is it doing right? I really hope every company, every every product that's being built that the people behind it are being mindful of that and are thinking about you know, what data are we using?

What system are we building? What evaluations are we conducting? What guardrails are we putting in? How do we communicate with our users and customers? How do we work with regulators so that when the rubber hits the road that we are you know, being responsible. I do believe a lot of this work is happening. It's not happening in the theater, to be honest. For example, So many pharmaceutical and healthcare industry companies are incorporating AI. literally I just came from the hospital to come to your... to your panel because I have a family member about to get a surgery in the next one hour or so.

I was just in Center Hospital looking at where AI is already being used and where AI could be used. And it's already happening. Doctors are using AI to to help them with charting. Radiologists, are using AI to assist them reading the MRI and the CT scans I do hope that we have more AI to help our nurses, to help family members, I got this long radiology report last night, and the first thing I did is send it to an AI so that they can help me to explain it. So all this is happening Safety measures are happening.

but there needs to be more in the right way, in a scientifically grounded way And that's the conversation that should be taking place instead of when you say the theater. Well, thank you for coming and I hope your person is okay. We all do. Um... The backlash is... It's being called the AI hate wave. I'm sure you've seen the video of former Google CEO, Eric Schmidt, getting booed at a college graduation. You spend a lot of time with students. What are they saying? and if they're scared, are the fears. justified.

Yeah, I do spend a lot of time with students. To be fair, my students are pretty privileged because they're Stanford students. I think it's I think it's even more important. And I try to do it myself that we spend time with our teachers with our nurses. with our parents, grandparents, and that's actually something I try to do. I try to talk to K-12, I try to go to the university places and talk to people where they feel that they're not part of the conversation. even Stanford students reflect some of this mixed sentiment.

There is anxiety, there is a sense of hope, there is also excitement There is also confusion. There is also um... simultaneously a sense of dignity and agency when AI can help me do things that I couldn't do before, and a sense of loss of dignity and agency if AI is is it gonna take my job? So I think the sentiment is mixed, and I really want to point out a lot of this sentiment. happens when there is a vacuum of thoughtful public discourse. Right now, the oxygen the air is off sucked into the polarized extreme of dumerism or total utopian.

And when hype takes all the oxygen in the room that void bruise the kind of anxiety. And it's actually that void we really need to care about because that's where real people live. That's where real people are seeking answers. And I think it's -- I'm as a scientist and an educator and an entrepreneur, I'm on ground zero. with students with educators with entrepreneurs, And I really do believe it's one of my responsibility to not hype and try to speak with with both sides and humility and and inspire people to recognize this is a technology that can truly empower a lot of our work and life can truly realize help us you know, have a better health care system, have better scientific discovery, have better environment, better education if we do the right thing.

We're both moms. We both have young teenagers. How do you think AI will change learning in the college experience. AI must change learning. AI must change K to 16 learning. I think this is one of the biggest opportunity for humanity in the next decade to come. is that what The most precious of our entire world. is human capital. Amen. And when we have gotten the technology that can answer standardized tests. whether it's It's a common core kind of test all the way to international Olympiad math exams When AI can do better than average human, It's not about Humans are bad.

It's about we need to change the education system. We need to change how we evaluate. We need to change the way we empower teachers to teach, to educate the next generation of students where they can use these tools be in power and do things that we can never imagine. - So do you think our kids will still learn? Absolutely, if we teach them right, if the society prepares them right, They should not be, all of the kids today should not be scared of AI. They should feel the human agency. to to lead AI, to use AI in the right way, and to use AI to make the impact that they want to make.

for the world. Thank you. Anthropic CEO Dario Amadei has suggested AGI is two to three years out. We'll get there by scaling the current paradigm. Dennis Hassabis, says we're at the foothills of the singularity, You said you don't even engage with the term. AGI. Are they wrong or is the disagreement about what we're calling the goal? I don't engage with the term AGI because the founding fathers of artificial intelligence as a scientific field had this dream of thinking and doing machines And that is a scientific quest. And that quest has been my lifelong career.

And I'm still on that quest. Now I'm combining that scientific question making products that can make people's life better. and that is the field called artificial intelligence. And, um, I'm okay people call it whatever they want, they can call it an apple, that's fine. I am focusing on building a technology that can truly that can truly make a difference in people's lives. a work. What's the one thing you'll have shipped? this year that we'll be talking about next year. I hope that we will be shipping a... model for spatial intelligence that will inspire incredibly exciting product opportunities that people haven't seen before.