OpenAI Researcher Dan Roberts on What Physics Can Teach Us About AI

[00:00] In the 40s, the physicists went to the Manhattan Project, even if they were doing other things. That was that was the place to be. And so now [00:07] AI is the same thing and basically set OpenAI as that place. So maybe we don't need a public sector organized Manhattan project, but it can be OpenAI. [00:20] *music* [00:37] Joining us for this episode is Dan Roberts, a former Sequoia AI fellow who recently joined OpenAI as a researcher. [00:44] This episode was recorded on Dan's second to last day at Sequoia before he knew that he would go on to become a core contributor to O1, also known as Strawberry. [00:52] Dan is a quantum physicist who did undergraduate research on invisibility cloaks before getting a PhD at MIT and doing his postdoc at the legendary Princeton IAS. [01:02] Dan has researched and written extensively about the intersection of physics and AI. [01:07] There are two main things we hope to learn from Dan in this episode. [01:10] First, what can physics teach us about AI? [01:13] What are the physical limits on an intelligence explosion and the limits of scaling laws? And how can we ever hope to understand neural nets? [01:20] And second, what can AI teach us about physics and math and how the world works? [01:25] Thank you so much for joining us today, Dan. Thanks. Delighted to be here on my probably second to last day at Sequoia, depending on when this airs and how you're going to talk about it. You will always be part of the Sequoia family, Dan.

[01:40] Thanks, I appreciate it. Maybe just to get started. [01:44] Tell us a little bit about who is Dan. You have a fascinating backstory. [01:48] I think you worked on invisibility cloaks back in college. Like what led you to become a theoretical physicist in the first place? [01:56] Yeah, I think [01:57] And this is my stock answer at this point, but I think it's true. I was just an annoying three year old who never grew up. I asked why all the time. Curious. How does everything work? I have a I have a 19 month at home right now, and I can see the way he followed the washing machine repairman around and had to look inside the washing machine. So like, I think I just kept that. [02:18] going and [02:20] when [02:21] when you're more quantitatively oriented than not, rather than going to philosophy, I think you sort of veer into physics. And that was sort of [02:28] What interests me? How does the world work? What is what is all this other stuff that's out there? [02:34] The question that you didn't ask, but maybe I'll just answer it ahead of time. The inward facing stuff felt less quantitative and more in the realm of the humanities. What is all this stuff? That's pretty physics-y. What am I? Who am I? What does it mean to be me? That felt not very science-y at all. [02:53] But with AI, it sort of seems like we can think about [02:57] both what does it mean to be intelligent and also what is all this other stuff and sort of [03:01] some of the same frameworks and so that's been very exciting for me. [03:05] - So we should be trying to recruit your 19 month old right now, is what you're saying? - Oh yeah, absolutely. He's got his, he grew out of one of his Sequoia onesie, but he fits into a Sequoia toddler t-shirt now. And so he definitely is ready to be a future founder. - I guess at what point did you know that you wanted to think about AI? At what point did that switch start to flip?

[03:26] Yeah, so I think like many people, I... [03:30] When I discovered computers, I wanted to understand how they worked and how to program them. In undergrad, I took an AI class and it was very much good old fashioned AI. [03:39] A lot of the ideas in that class actually are coming back to be relevant. But at the time, it was, it seemed not very practical. It was a lot of, you know, if this happens, do that, you know, and there was also some, some game playing there that was sort of interesting. It was, it was very algorithmic, but it didn't seem related to what it means to be intelligent. Can we just. [03:59] pick up on that real quick? Like what do you think it means to be intelligent? [04:02] That's a great question. [04:04] I can see wheels turning. [04:08] Yeah, well, this is one of those questions where I don't have a stock answer, but I think it's important to not say nonsense. [04:18] One of the things that's exciting to me about [04:21] AI [04:22] is... [04:23] the ability to have systems that do what humans do and to be able to understand them from [04:30] what are the lines of Python that caused that system to do something to [04:36] you know, to trace that through and understand what are the outputs and, you know, how the system can like see and classify what it means. You know, what is a cat? What is not a cat or, or can write poetry? And, you know, it's just a few lines of code. Whereas if you're trying to study humans and ask what does it mean for humans to be intelligent, you have intelligent, you have to go from

[04:56] biology through neuroscience through psychology up to [05:00] other higher level versions of ways of approaching this question. And so I think maybe a nice answer for intelligence [05:10] at least the things that are interesting to me is the things that humans do. [05:14] And and then now if you up, [05:17] pull back the answer that I just said a second ago. [05:20] that's how I connected to AI. AI is taking pieces of what humans do and we're understanding at least an example that's kind of simple and easy to study that [05:31] that we might use to understand better what it is that humans do. [05:35] So Dan, you mentioned when you were studying AI in college, there's a lot of what sounds like kind of hard-coded logic and brute force type approaches. Yeah. Was there a moment that kind of clicked for you of like, oh, okay, this is different. Was there a key result or a moment where it was like... [05:51] okay, we're going bigger places than kind of the if this, then that logic of the past. [05:56] Yeah, actually it didn't click. I sort of wrote it off. And it would be great if there was a separation between like 10 years between then and the next thing that I'm going to say. But actually, the writing off maybe lasted a year or two, because then I went to the UK for the first part of grad school. I spent [06:14] very long time in grad school and I discovered, I discovered machine learning and, and, and, and, and a more statistical approach to artificial intelligence where you

[06:27] have a bunch of examples of large amounts of data or at the time, [06:32] maybe we wouldn't have said [06:33] large amounts of data, but we would have said, you know, you have, you have data examples of a task that you want to perform. And I mean, there's different ways that machine learning works, but you, um, [06:42] write down a very flexible algorithm that can adapt to to to these examples and start to perform in a in a way similar to to to the examples that you have and and that approach [06:54] borrows a lot from physics. It also, at the time I started, so graduated college in 2009, discovered machine learning in 2010, 2011, 2012 is the big year for deep learning. And so, you know, there's not a big separation here between write off and, and rediscovery, but I think, and machine learning clearly existed in 2009. It just wasn't related to the [07:15] class I took. But this approach made a lot of sense to me. And it started to have, you know, I got lucky in that it started to have real progress and [07:22] and seemed to fit in a framework that I understood scientifically, and I got very excited about it. [07:27] Why do you think there are so many people [07:30] who come from a similar background or similar path to you? Like a lot of ex-physicists working on AI, like... [07:36] Is that a coincidence? Is that herd behavior? Or do you think there's something that's particularly... [07:41] that makes physicists particularly well suited to understanding AI. [07:45] I think the answer is yes to all the ways you... All the above? [07:50] infiltrate lots of different subjects and we get parodated about the way that we go about trying to use use our hammers to to tackle these things that may or may not be nails throughout history there are a lot of times that physicists have contributed to things that look like

[08:02] machine learning. I think in the [08:04] near term, [08:05] the path that physicists used to take when they don't go, didn't, [08:10] remain in academia often was going into quantitative finance, [08:14] than data science. And I think [08:16] Machine learning was [08:18] and its realization industry was exciting because, again, it's something that feels a lot like actual physics. And, [08:27] is working towards a problem that's very interesting and exciting for a lot of people. I thought, you know, you're doing this podcast because you're excited about AI. Everyone's excited about AI. And in many ways, it's a research problem that feels a lot like the physics that people were doing. [08:42] But I think the methods of physics actually are... [08:46] different from the methods of traditional computer science and very well suited for studying [08:52] large scale machine learning that we use to work on AI. There's [08:58] Traditionally, physics involves a lot of interplay between theory and experiment. You have come up with some sort of model that you have some sort of theoretical intuition about. Then you go do a bunch of experiments and you validate that model or not. Then there's this tight feedback loop between collecting data, coming up with theories, [09:15] toy models understanding moves forward by that. And, you know, we get these really nice explanatory theories and I think the [09:23] the way that [09:24] big deep learning systems work, you have this tight feedback loop where you can do a number of experiments. The sort of math that we use is very well suited to the math that a lot of physicists are familiar with. And so I think, I think it's very natural.

[09:36] for a lot of physicists to work on this. And those tools, a number of them differ from sort of the traditional [09:44] at least theoretical computer science and theoretical machine learning tools for studying the theoretical side of machine learning, and maybe also differentiation between just being an awesome engineer and also being a scientist. And there's tools from [09:56] doing science that are helpful in studying these systems. Dan, you wrote this, what I thought was a beautiful article, Black Holes and the Intelligence Explosion. And in there, you talk about this concept of sort of the microscopic point of view, and then the system level point of view, and how physics really equips people to think about the system level point of view. And that has a sort of complementary benefit to understanding these systems. Can you just take a minute and kind of explain sort of microscopic versus system level, [10:26] helps to understand the system level. [10:28] Sure. So let me start with an analogy that I think is very-- goes even further than an analogy. [10:38] going back [10:39] what year is it? Maybe like 200 years or so. There's around the time of the Industrial Revolution, there was [10:45] steam engines and steam power and a lot of technology that resulted from this and ultimately powered industrialization. [10:52] And [10:54] In the beginning, there was a lot of engineering of these steam engines. And there was this high level theory of how this work called thermodynamics where [11:03] I... [11:04] imagine everyone's seen this in high school, perhaps, where, you know, there's the ideal gas law that tells you that there's some relationship between pressure at volume and temperature. And, you know, these are very macro level things, like you can buy a thermometer, you can also like measure the volume of your room, and you can buy a barometer as well, maybe people don't

[11:25] or look up on the weather report. But these are like measurements that we use and we talk about. [11:30] But then [11:31] underlying this, and it took us a little bit later to like validate this and understand it, there's the notion of atoms and molecules, the air molecules bouncing around and somehow [11:43] we now understand that those air molecules give rise to things like temperature and pressure and [11:48] volume I guess is [11:50] it's easier to understand that the gases, the molecules are confined to room. And, and, but there's a precise way in which you can start with a statistical understanding of those molecules and derive thermodynamics, like derive the ideal gas law from it. And you can go further than that to drive, you know, that it's ideal because it's wrong. It's, it's just a toy model, but you can, there are corrections to it and you can sort of understand, you know, from the microscopic perspective, which is the, the molecules, which we don't, we don't really interact with. [12:20] aggregate properties give rise to sort of this [12:23] this behavior that we do see at the macro scale. And part of [12:29] To get to your question, I think there's a similar thing going on with deep learning systems. I wrote a book with Shoyeda and Boris Hanin on [12:40] on how to apply these sorts of ideas to to to deep learning and at least in an initial framework that that allows you to start doing this in an initial way. And and to answer your question, [12:51] we [12:53] the sort of micro perspective is you have neurons and weights and biases and we can talk about in detail how that works but you know when people think of the architecture there's there's some very

[13:04] specific, some people say circuits, there's specific, um, ways in which these, these things, um, you know, there's an input signal, which might be an image or text, and then there's many parameters and, and, you know, there's a very, it's very simple to write down. It's not that many lines of, of, of code, even taking into account the machine learning libraries, but it's, you know, it's like a very simple set of equations, but there's a lot of weights. There's a lot of numbers that you have to know in order to get it, to do something. [13:29] And that's sort of the micro-- that's like the molecules perspective. [13:32] And then there's there's like the the macro perspective, which is, well, what did it do? Did it did it produce a did it produce a poem? Did it produce? Did it solve a math problem? How does you know how do we go from that those weights and biases to that macro perspective? And so for for for statistical physics, the thermodynamics, we understand that completely. And you know, you can imagine trying to do the same sort of thing, literally applying the same sorts of methods in to to [13:59] to understand how does the underlying micro statistical [14:04] behavior of these models lead to the sort of macro, or as you said, system level perspective. [14:12] Dan, maybe speaking of scaling laws, and I think you were at our event, AI Ascent. Andre Karpathy mentioned that current AI systems are like five or six orders of magnitude off in efficiency compared to biological neural nets. Yep. [14:27] How do you think about that? Do you think scaling laws get us there? Just a combination of scaling laws plus hardware getting more efficient or like?

[14:35] Do you think that there's kind of big step function leaps that need to come in research? There's maybe two things that could be meant here. One is that [14:44] the way humans [14:46] seem to work at a similar scale to AI systems is much more efficient. You know, the amount of, we don't need to see trillions of tokens before we speak, we see a much [14:56] You know, my toddler is already starting to speak in sentences, and he's been exposed to far less tokens than a typical large language model. And so there's some sort of disconnect between human efficiency at learning and what large language models do. Of course, they're very different systems are designed. You know, the way in which they learn is right now very different. And so in some sense, that's to be expected. So there's this gap here that you could imagine bridging. There's another thing that I think is not what you meant, but I think is, [15:26] sort of the thing to answer about with respect to scaling laws, which is, [15:29] And I talked about this a bit in the article, [15:32] a lot of people seem to talk about this, which is what is the final GPT? You know, there's GPT four right now, and it could be other companies as well. But since I'm going to join OpenAI, let me represent my, my, my new company, right? So is it going to be six is going to be seven at some point, right? If assuming we have to scale things up, there are things that are going to break, whether they're economic, we're going to run out of [15:54] you know, we're going to try to train a model that's larger than the world's GDP or GWP, whatever the, however the D works for the world. And, or we're going to run out of, you know, we're not going to be able to produce enough GPUs or we're not going to be able to put, you know, it's going to cover the surface of the, you know, a lot of these things are going to break down at some point. And so probably the economic one happens first. So how many, you know, how many

[16:24] practically and where does that get us? [16:26] And then I think... [16:27] I think to tie those two perspectives together, there's-- so scaling on its own-- and of course, it's impossible to disentangle this because people are making things more efficient. But you could imagine there's the take literally what GPT2 was, which was the initial big model, and keep scaling it up. [16:43] is that going to get us to some super different, exciting, economically power, or however you want to define what the end state of AI research and AI startups and AI industry is? [16:57] Um, or do we need lots of new, exciting ideas? And, and again, of course you can't really just disentangle these, but I think the, the general scaling hypothesis is that it's just the scaling and it's not the ideas. [17:07] that matter. [17:08] Um, whereas the, how do we get to efficient like humans, I think requires like [17:13] non-trivial ideas. And to answer your question, [17:17] The reason I'm excited about joining OpenAI is that I think there is high leverage to be had in the ideas, you know, in going beyond scaling and that we will need that in order to get to the next steps. And I have no idea what I'll be working on. But when this airs, I guess I will know what I'm working on. But, you know, that's what's really exciting to me. [17:47] Like Transformers came out, great idea. Since then, we've largely been in this race to scale. It feels like things are starting to asymptote.

[17:56] for a bunch of practical reasons that you mentioned. Is the pendulum swinging back toward ideas as the currency? You know, it's less now about who can, you know, have the biggest GPU cluster and more about finding new architectural breakthroughs, whether that's, you know, reasoning or something else? [18:11] Yeah, that's a really great question. [18:15] There's this [18:17] article by Richard Sutton, um, [18:20] called the bitter [18:22] lesson, a bitter pill. And [18:24] and [18:25] It basically gives the argument that [18:29] that ideas aren't important, that scale is what you need, and that all the ideas are always trumped by scaling things up. And that's a bunch of things, but maybe that's a high level takeaway. [18:42] And there's a sense of this where there are a lot of interesting ideas that came out in the '80s and '90s that [18:51] people didn't really have scale to explore. And then I remember when after AlphaGo and DeepMind was, was writing a lot of papers, people were rediscovering those papers and re-implementing them in deep learning systems. But this was sort of still before people realized, no, the thing that you need to do is scale up. And, and even now with Transformers, people are exploring other architectures or even [19:10] simpler architectures that we that we knew before that seemed to be able to you know there's a notion you know maybe scaling laws don't come from [19:17] as long as the architecture isn't sick in some way, they come from sort of the underlying data process and having large amounts of data rather than from [19:26] from having a special idea.

[19:28] I think the real answer is that there's a balance between the two, that scale is hugely important. And maybe it was just not understood how important. And we also didn't have the resources to scale things up at various times. You know, the things that have to go into producing these GPU clusters that are producing these models are, you know, you guys know this as well, but like there's a lot of parts along the supply chain or along the product chain, whatever. [19:53] you actually call it, in order to make those things happen and deploy them. And even the way GPUs were originally, they've now coevolved to be-- [20:00] well suited for these models. [20:04] in some sense you can think of transformers was a good idea, was because it was designed to be well suited to train on the systems that we had at the time. And so sure, these other architectures could do it at an ideal scientific level, but at a practical level, [20:19] it was important to get something that was able to reach that scale. So I think, [20:25] you know, if you brought in ideas to... [20:28] to be that sort of thing that's that's married with scale in some way then um [20:33] I mean, I still think ultimate, like, you know, someone came up with the idea of deep learning. That was an important idea. You know, there's, [20:39] Pitts and McCullough came up with the original idea for the neuron, and then there's lots of [20:45] Rosenblatt came up with the original Perceptron. And there's like a lot of people from going back [20:50] like 80 or so years of people making important discoveries that were ideas that contribute. So I think it's both. But it's easy to see how, you know, if you get to apply

[21:03] If you're bottlenecked and then all of a sudden you get to apply, if you're bottlenecked, people think about ideas. And then if you unlock a new capacity of scale somehow, then you just see a huge set of results. And it seems like scale is super important. And I really think it's more of a synergy between the two. Maybe on the topic of the race to scale, Dan, you mentioned kind of just the economic constraints and realities, which I guess are... [21:24] more practically a ceiling in the private sector. You also mentioned the Manhattan Project earlier in terms of things that physicists have been involved with. [21:33] Do you think we need a Manhattan Project style thing for AI, like at the nation state or at the international level? [21:39] - I think it's a good question. [21:40] Um, well, [21:42] One thing I can say is that part of the process that led me to open AI, as I was talking with your partner, Sean McGuire, who brought me to Sequoia in the first place and trying to figure out, is there a startup that makes sense for me to for me to work on that has the right mix of [21:58] sort of scientific questions, research questions, and also as a business. And I think it was Sean that said, and I don't mean the analogy in terms of the negative impact of what the [22:11] You might think of the Manhattan Project, but just in terms of the scale and the organization, he said, you know, [22:16] In the 40s, the physicists went to the Manhattan Project, even if they were doing other things. That was that was the place to be. And so now. [22:24] AI is the same thing and basically set open AI as that place. So maybe maybe we don't need a public sector organized

[22:33] Manhattan Project, but it can be OpenAI. OpenAI is the Manhattan Project. I love that. Well, maybe that's not a direct quote that we want to be taking out of context, but I think in terms of- It's the metaphorical Manhattan Project. Yeah. In terms of scale and ambition, in terms of, I think a lot of physicists would love to work at OpenAI for a lot of the same reasons that they probably were excited to [22:56] Well, [22:56] Okay, there's a number of different reasons. Maybe we just have to leave it as a nuanced thing rather than making broad claims. [23:03] Can we talk a little bit about this? [23:07] Can we ever understand? [23:09] especially as we go to these deep neural nets, or do you think it's... [23:13] a hopeless black box, so to speak. [23:15] Yeah, I think within the this is this is my answer to the what are you a contrarian about? Although maybe, you know, on the Internet, everyone takes every side of every position. So it's hard to to to say you're you're having a contrarian position. But I think within AI communities, you know, I think of my contrarian position is that we can really understand these systems. [23:34] And physics systems are extremely complicated. [23:38] and we have made a huge amount of progress in understanding them, I think these systems sit in the same framework. And, you know, another principle that [23:47] that Shona and I talk about it [23:48] in our book, and that's a principle of physics, is that there's often extreme simplicity at very large scales. Basically, due to the statistical averaging, or more technically the central limit theorem, things can simplify. And I'm not saying this is what happens exactly in large language models. Of course not. But I do think that we can apply sort of the methods that we have, and also maybe hopefully have--

[24:13] AI that can help us do this in the future. And by AI, I mean tools, not like individual intelligence is just going, running on their own and solving these problems. But but I guess I feel at the extreme end that that that this is not [24:26] going to be an art that the science will catch up and that it will be able to make extreme leaps in really understanding how these systems work and behave. So Dan, we've talked a bunch about what physics can teach us about AI. Can we talk a bit about what AI can teach us about physics? Are you optimistic about domains like physics and math and [24:47] how these emerging models can, you know, probe further into those domains. [24:52] Thank you. [24:53] Yes, I'm definitely optimistic. [24:56] my, um, [24:58] My perspective is that math will be easier than physics, which maybe betrays the fact that that that I'm not a mathematician. [25:09] And I'll say I can say I can give explain why I think that in a second. [25:18] But, you know, I still have a lot of friends that work in physics and they, you know, there's a... [25:25] There's like a growing sense and maybe even approaching a dread that, and maybe this is actually the answer to why do physicists work on AI? Because, you know, if what you care about is the answer to your physics question and you want to make it happen as soon as possible, what is the highest leverage thing you can do? Maybe it's not work on the physics question you care about, but it's work on AI to make the, you know, to, because you think that the AI might end up solving those questions very rapidly anyway.

[25:55] but I think within the... [25:56] the theoretical physics community that I come from, that this is sort of a [26:00] a thing that someone gets thrown around and discussed. I think maybe to give a more object level answer, I think what's exciting about math and [26:11] Maybe when you have Noam Brown on, if you have him on, he'll talk about this, but this is something that he's talked about for a while before he joined OpenAI. [26:21] I think that we have... [26:25] You know, we made a lot of progress in terms of, um, [26:29] solving games by doing more than just looking up what is the strategy that we should use to play the games but also being able to simulate forward and you know you know the way the way that if i'm in a very hard position in a particular game rather than just playing with intuition i might sit and think about what i should do yeah sometimes this goes under the name inference time compute rather than training training time compute or pre-training [26:52] and [26:53] you know, there's a sense in which what it means to do reasoning is very related to this ability to sit and think. [27:00] So we know how to do it for games because there's a very clear witting and loss signal. So you can simulate ahead and sort of figure out what it means to do good or not. [27:07] And I think math in some parts of math, again, I'm not a mathematician and, well, you know, always scared about talking about math publicly and saying something wrong to all the upset mathematicians. But it seems like certain types of math problems are not as constrained as games but are still constrained enough where there's a notion of, you know, like...

[27:27] finding a proof, right? There's different problems in terms of search, in terms of how do you figure out what is the next move in the proof. But the fact that we might call it a move suggests that there's things in math that feel a lot like games. And so we might think that the fact that we can do well at games maybe means that we can do well at certain types of mathematical discovery. [27:47] Well, I was going to say, since you mentioned Noam, he likes to use the example with test time compute of whether it could help to prove the Riemann hypothesis. Right. Is there a similar problem or hypothesis in the world of physics that you are optimistic AI can help to solve sometime in our lifetimes? [28:06] Yeah, so I mean there's a Millennium problem relating to physics and if I try to [28:10] um, [28:11] remember exactly what it is, I'm sure I will [28:14] Butcher it and then no one will believe that I'm actually a physicist, but it's related to the, you know, it's a mathematical physics question related to the Yang Mills mass gap and. [28:23] But I think what I wanted to say is that I think some of the flavor of what physicists care about in doing physics feels a little different. This is where I might get in trouble. It feels a little different than some of like the mathematical proof type things. Physicists are known to be more informal and, you know, hand wavy, but also on the other hand connected to, in some sense, connected to the interplay between experiment and the sort of models that physicists study is maybe what saves [28:53] them is that they have things that are informal and hand wavy, but very explanatory. And then the mathematicians-- it's like we were saying earlier, the engineers discovered all the exciting industrial machines, and then the physicists maybe cleaned up a bunch of the theory about how that works. And then the mathematicians come later and clean up, formalize everything and clean it up even more. And so there's mathematicians or mathematical physicists that clean up a lot of make proper and try and--

[29:19] understand in informal ways some of the stuff that physicists do. But [29:24] Rather than talking about-- I think the-- [29:28] the key point there is that the sort of questions that are interesting to physicists maybe don't look like proofs but maybe they look like and maybe it's not about how do we [29:37] given a, a particular model, how do we actually solve it? Like once, once things are set up correctly, like. [29:42] it's often, you know, senior or, you know, people that are trained in the field are able to, to sort of figure out how to analyze the systems. It's more the other stuff, like what is the right model to study? Does it capture the right sort of problems? Does this relate to the thing that you care about? What are the insights you should draw from it? And, and so [29:59] for AI to help their [30:01] I think it would look different than the way we're sort of trying to build AI systems for math. So rather than here's, you know, here's the word problem going, you know, solve, solve, solve this high school level problem or, you know, prove the Riemann hypothesis. It's like, you know, the questions in physics are like, what is quantum gravity? What happens when something goes into a black hole? And that's not like, you know, start generating tokens on that. Like, what does, what does that even look like? [30:26] And if you go to a physics department, people hang out at the Blackboard, they chat about things. Maybe they sketch mathematical [30:34] things but there's a lot of other things that go into this. So maybe that data that you need to collect looks more like that. Or maybe it looks like the random emails and conversations on Slack and the scratch work. [30:48] And so [30:48] I mean, there are definitely tools that we can use, like,

[30:52] help me understand this new paper so I don't spend two weeks trying to study it and understand it. Maybe let me ask questions about it. I think there are problems with the way that's currently implemented, but I think there are a lot of tools that will help accelerate physicists, just like Mathematica, which is a software package that does integrals, and it does a lot more than that. Sorry, Stephen Wolfram, but I use it to do integrals. Sometimes it doesn't know integrals, and you can look... [31:22] them up in these integral tables and [31:25] Anyway, um... [31:28] You know, I think, you know, there's-- and this applies to other branches of science, too. Like, I think there are the ways in which the questions are asked and what it means to do science in different fields maybe looks [31:39] can look further and further from gains, let's just say. And so to the extent that that's true, I think we'll need to [31:44] And not even clear that we'll need lots of ideas. I mean, sorry, we will need lots of ideas, but it's more just like I think we'll just have to approach them all differently. And maybe eventually we'll have a universal thing that knows how to do all of it. But initially, at least to me, a lot of these things feel a little different from each other. [32:02] - Thank you. [32:03] You'll have a front row seat to it, in part because you're also on the prize committee for the AI Math Olympiad, which is something I'm personally super interested in. Maybe to your last point, I'm kind of like maybe eventually this stuff generalizes. Like, why do you think people are so focused on solving the hardest problems today? Like, [32:18] Physics, math, those were the subjects that everyone was terrified of in school, right? Where it feels like there's a lot more other domains that are also unsolved for now. Do you think going for the hardest domains first...

[32:30] kind of lets you get towards a generalized intelligence. Like how does solving these different domains [32:36] kind of fit together in the grander puzzle. [32:39] - Yeah, the first thing that comes to mind when you said that is to just push back and say, well, it's not hard. These are the easy domains. [32:46] I mean, I'm... [32:47] I'm bad at biology, but it doesn't make any sense to me at all. My girlfriend actually is bioengineering and in biotech, and so what she does just makes no sense to me. Can't understand any of it. Where physics makes complete sense to me, [33:04] I think... [33:06] I think maybe a better answer or a less global answer is that [33:11] Um, like I was trying to say about math, their, their constraints and, you know, and in particular with math, a lot of it is unembodied. You don't have to go and do experiments in the real world. You know, they're, they're sort of self-sufficient and that's, that's close to like what generating texts like. [33:25] the way language models work or even the way some reinforcement learning systems [33:30] work for for games. And so I think the further that you go from that, the messier things become, the harder it probably probably is, then also the harder it is to get the right kind of data to train these systems, if you want to build a [33:41] AI and people are trying to do this, but it seems difficult if you want to build an AI system that solves biology. [33:48] I guess you need to [33:49] also make sure robotics works so that it can do those sorts of experiments and it has to understand that sort of data. Or maybe it has humans do it. But there's a lot, for a self-sustaining AI biologist, it seems like there's a lot of things that are going to go

[34:03] into it. I mean, on the way, we'll have things like AlphaFold3 [34:07] which just came out and which I didn't get a chance to read the details of. But I saw that they were trying to use it for drug discovery. And so I think [34:15] each of these fields will have things [34:17] developed along the way, but I think the... [34:21] the less constraints there are and the messier and more embodied it is, the harder it will be to accomplish. That makes sense. Hard for a human is not the same thing, doesn't correlate to hard for a machine. [34:31] Yeah, plus also maybe humans disagree about what's hard or not. Some of us think more like machines, I guess. And then I guess the second question was like, do you think it all coalesces into like one big model that understands everything? Because right now it seems like there's a lot of domain-specific problem solving that's happening. Yeah. Yeah. [34:47] I mean, the way things are going, it seems like the answers should be yes. [34:50] It's really dangerous to speculate in this field because everything... [34:53] you say is wrong. Um, usually much sooner than we're on record. We'll hold you to it. Yeah, exactly. [35:00] Um, [35:01] But also, what does it mean to be differences? There's a trivial way to make both things-- make the question meaningless by, you say the model is the union of all those other models. But there's also a sense in which mixture of experts is not-- [35:14] was originally meant to be that. It's not that in practice at all. But there's a sliding scale here. But it does seem like people-- [35:22] um [35:23] at least the big labs are going for the one big model and have a belief that that's [35:28] I don't know, but maybe I will in the future. [35:31] understand what the philosophy is there.

[35:34] Yeah. [35:35] Dan, we have a handful of more general questions to kind of close things out here. So I'll start with a high-level one. [35:41] if we think kind of short-term, medium-term, long-term and call it [35:45] you know five months five years five decades [35:48] What are you most excited or optimistic about in the world of AI? [35:54] Five years ago was about... [35:58] I was after the transformer model came out, but it was around maybe when GPT2 came out. So it seems like, you know, for the last five years, we've been doing scaling, I imagine, within the next five years, we'll see that [36:09] scaling will terminate and maybe it will terminate in, you know, a utopia of some kind, you know, that the people are excited about where we're all post economic and so forth and we'll have to, you'll have to shut down all your funds and [36:21] you know, return monopoly money because money won't matter or, you know, or we'll see that we need lots of ideas. Maybe there will be another AI winter. I imagine that [36:31] Um, and again, scary to really speculate, but I imagine like something [36:36] Interestingly will be something will be interestingly different within five years about AI and it might just be that AI is over and we're on to the you know, we're on to the next exciting investment opportunity and you know, the everyone else will shift elsewhere. And, you know, I'm not saying that that's not what's motivating me about that AI, but you know, so maybe five years is enough time to see that and, um, [36:58] I think in [37:00] In one year, I mean, or [37:03] There was a five. I messed this up. Whatever. Maybe it was five months. I don't remember. Five months. It was five months. It's okay. It's okay. These are approximations. I know you said physicists are very hand-wavy. Venture capitalists are very hand-wavy. These are approximations. Yeah. In physics, I like to joke that there's like three numbers. There's zero, one, and infinity. And those are the only numbers that matter. Things are either arbitrarily small, arbitrarily large,

[37:33] But yeah, for five months, I mean, I'm excited to-- [37:37] well, to learn what's exciting at the forefront of a huge research lab like OpenAI. And I think one thing that will be interesting will be the delta between... [37:49] The next. [37:51] generation of models, right? Because there's ways in which things are scaling up in terms of, and it's not really public, I guess, aside from meta, but in terms of size of data, size of models, and we see scaling laws. And scaling laws, though, relate to something like the loss and [38:07] And... [38:08] you know, it's hard to translate that into actual capabilities. And so what will it feel like to talk to the next generation model? What will it look like? Will it have a huge economic impact or? [38:17] or not. [38:20] I, you know, [38:21] in terms of like estimating velocity, right? You need a few points. You can't just have one point and we sort of have, [38:27] you know, we're starting to have that with GPT three to GPT four, but you know, I feel like with the next Delta, we'll get to really see what the velocity looks like and what it feels like going from, from model to model to model. And maybe I'll be able to make a better prediction in five months from now, but then I guess I probably won't be able to tell it to you guys. [38:44] Thanks, Dan. One thing that stood out to me is just like, your writing is so accessible and light and funny. [38:53] And that's not what I'm used to when I read super technical stuff. Like, [38:57] Do you think all technical writing should be informal and funny? Is that deliberate?

[39:03] It's definitely deliberate. It goes into this, I think in some sense, it's inherited, I mean, I definitely am a not serious person. But I also think it's inherited sort of from the style of the field that I that I came from. But I'll tell you a story. I was I was at lunch, I was a postdoc at the Institute for Advanced Study in Princeton. And I was having lunch and joking around with this Professor Nadi Seiberg, who's a professor at the Institute. And [39:30] And then we got into, I think we're talking about, someone asked a question about what is a good title, and I was like, oh, the title has to be a joke, and he was on board with that, and then I was explaining that [39:42] For me, the reason to write a paper is for the jokes. You have a bunch of jokes in mind, and then you want people to read those jokes, and so you have to package it into the science product, and people want to read the science product, and they're forced to suffer through the jokes. And Nadi, he's this Israeli professor, and he's like, I don't get it. Why can't you just do the science? Why do you need the... The jokes are great, too, but you should write for the science, not for the jokes. And I was adamant that I write for the jokes. But I think it's what you said that... [40:11] at some point you learn about the scientific method and the formal ways of doing things, and you learn all these rules, and then you grow up a bit. Or maybe I had a roommate who was a linguist, he's now a professor of linguistics at [40:24] at UT Austin, and he emphasized that you can [40:28] He would tell me which rules that I could break or where the rules come from and why they're important or not. You sort of realize that

[40:34] that you can break these rules and the ultimate goal should be [40:38] is the reader going to read it and understand it and enjoy it. So you don't want to do things that compromise their ability to read and understand, but you don't want to obscure things, you want to make it and you know, if it's more enjoyable, people are more likely to read it and take the point. It's also more fun if you're if you're writing it. So so I think [40:53] I think that's where that comes from. [40:54] Dan, thank you so much for joining us today. We learned a lot. We enjoyed your jokes. And I hope you have a wonderful second to last day at Sequoia. Thank you for spending part of it with us. We really appreciate it. [41:07] Thanks. I was absolutely delighted to be here chatting with you guys. It was wonderful. [41:35] Thank you.