#1073 David Wolpert: History as a Stochastic Process

00:00 Ricardo Lopes: Hello, everyone. Welcome to a new episode of the Center. I'm your host, as always, Ricardo Lopton today I'm joined by Doctor David Wolpert. He's professor at the Santa Fe Institute. Most of his work is in thermodynamics or computation, foundations of physics, dynamics of social. Organizations, machine learning game theory, and distributed optimization slash control. And today we're going to focus on a paper of his titled The Past as a stochastic Process. So, Doctor Wolpert, welcome to the show. It's a pleasure to everyone.

00:34 David Wolpert: It's a pleasure to be here. Very good to meet you and everybody else online. Yeah.

00:40 Ricardo Lopes: So, just to introduce the topic, what is a stochastic process?

00:46 David Wolpert: Hm, OK. So, um, a deterministic process is something that is governed, for example, by a differential equations. Um, IT is something where the state of a system right now and this, um, uh, definition of state might be somewhat elaborate, but the state of the system right now uniquely defines its future trajectory. There is no randomness at all. So, um, the idea of physics is that at the very foundation of physics, if you have a closed system, so it's isolated from the outside world, it is completely deterministic. If I were to tell you right now the precise positions of all the planets in the solar system, And the sun, along with their masses and their velocities, and the, um, if there were no gas to sort of add some randomness, then I could tell you the positions and velocities of the planets millions of years into the future. Everything is deterministic. It's bang, bang bang. Um, A stochastic process is the general, well, formerly in mathematics, what the term refers to is A general um uh mathematical structure that allows us to talk about trajectories into the future that, um, do not have that characteristic that they're all bang bang deterministic. There is inherent randomness. So, to give a simple example from the foundations of physics, um, if there were to be a quantum mechanical effects. That um have any kind of implication for the trajectory of the system we're interested in, then we have a stochastic process going into the future. But also more generally, if I don't have a system so precisely isolated from all other variables, and it's so small and easy to consider like the plants in the solar system, then we as humans. Phenomenologically it's a simple matter of practice. Um, CANNOT avoid having stochasticity in all of our, um, uh, predictions about the future process. Um, WHAT are the stock markets going to be tomorrow? What's the weather going to be in one week? What's going to happen with the new power in the United States starting up today? What are the implications for the whole geopolitical system within a month? Things like this. These all have somewhat randomness, um, uh, some degree of randomness, and also that randomness can grow. So in the most general sense, a stochastic process is a mathematics. That allows us to deal with, allows us to analyze and investigate and make predictions using such stochastic processes.

04:01 Ricardo Lopes: So, but we're going to focus on the history specifically here, which is what you do in your paper. So what kinds of methodologies do you apply when studying history?

04:12 David Wolpert: OK, yeah. So, um, for example, if we were predicting the future, that question about the geopolitical situation one month from now. Um, SO, History, so in the, in the papers you're referring to of mine, of mine and many others, um, we are considering what's called deep history over very, very long time spans, um, phenomenology that applies to all civilizations. So the amazing thing. Um, THIS was developed, um, originally by about a group of about 50 archaeologists, um, something called the Seshat data set. Seat was an Egyptian god of, I forget what, to be quite honestly. Um, AND what it is, is a recording over um century intervals, so the year 100, the year 200, the year 300, of about 50 different characteristics of A human society at about 30 or 40 different locations on all planets all over the world. And so it's a massive data set. It was a huge undertaking. The reason that so many different archaeologists and anthropologists and historians were involved, was because they had to all agree on a common set of standards for measuring these 50 variables. What they found astonishing, it, it almost can't be true, but it is, is that all civilizations across all time, all human cultures, there are very, very striking clear trends in these time series data sets. All civilizations, all humans do the same kinds of things over intervals of century to century. Where we're talking at the level of big um uh geopolitical societies, things like Latium, the uh plain of uh Rome, things like the um uh plain um that had uh um Kyoto, um. ERA Japan, things like um Mesoamerica, the Mayans, things like this, the um uh the um uh Khmer civilization, the Mekong Delta, the Mekong Rivers, those were all the geopolitical regions and the societies that existed there over these time scales followed these very striking trends. Bizarre. It's almost can't be true. It's like a joke from some science fiction movie, a grade B science fiction movie. So when I first encountered this, me and some of my collaborators, um, I didn't believe it. I dug into the statistics of this paper, um, because I have a background doing machine learning. I've done a ton of different things in the past. Um, AND, but I found that, nope, statistics is correct. These conclusions are valid, but it was a first pass. There are some subtle things that they had not looked into, or, or not so subtle, in fact, some of them. So this is where me and some of my collaborators, we start to investigate these phenomena, and we found, for example, the following elaboration. It turns out that all societies, when they start going, get getting going, so things like through the year 500 or so, ancient China, ancient Rome, so on and so forth, first, they grow in size without any increase in their ability to perform computation, where you measure a society, how well it can do computation. Um, IN terms of things like how sophisticated is the monetary system, how good is the transport system, how many levels of bureaucracy, and so on. At first, every single society grows in size without any increase in its ability to perform computation, then it stops growing in size, only increases in essence, how smart it is until it reaches a new threshold, and it's the same threshold for all societies. And then bang, now it can start increasing in size again. They all do this. So, how can we measure these things? This is a very roundabout way of answering your question, but I'm getting there. How can we um analyze this situation? One century intervals over say 56 1000 years for about 30 different societies, since these are 50 different variables, that is a teeny teeny teeny data set. It's far too small for the dimension of the space to use some of the standard time series techniques. You may have heard of things you or your um uh listeners, your viewers, like what are called auto regressive models, ARMA models, um, a nonlinear time series. These are the kinds of things that are used all over Wall Street and so on. Those require vastly more data than we have. So, what we're doing is we're um using some of the tools from the stochastic process literature. Which allow us to say that we can view um the um dynamics of all these societies in this 50 dimensional space, as though it were just a sum of two very simple processes. One is a deterministic trajectory. If my society is here, the deterministic component says we're going there in the space of characteristics of the society. But there's also Gaussian noise added to it. Wherever we end up in one century, that'll be a new position, so there's a new trajectory vector, and then we go in that direction where we add a little bit of noise, Gaussian noise to that. And we just keep doing this, a step by step by step where we add noise at each step. This has many fewer what are called degrees of freedom, far, far simpler to fit to the data set, and that's actually what we um have been considering in things like the paper that you're referring to, that more recent one, the past stochastic process. That is the underlying technical tool. It's called things like if, if anybody wants to look it up, it's called Fucker Planck um dynamics, F O K K E R and then Planck is the um the very, very famous um physicist, P L A N C K.

11:03 Ricardo Lopes: So you might have answered partly at least the question I'm going to ask now, but what is a stochastic process framework? How does it apply to the study of history and what can we learn from it?

11:19 David Wolpert: Yes, so the, um, what I just gave you is a simple example of of Farker-Planck dynamics where you have what's called a veenner process, um W I N E R. These are all famous. Mathematicians, scientists from about a century ago, um, where you have Gaussian, um, noise added to each step. There are many variations of this basic idea that you can do, where at each moment, the key feature of this that makes it so powerful and so easy to fit even when there's not much data, is that we are considering only the present. To figure out where to go next. We're not looking at the previous time steps as well. There's no hidden variables with extra memory, like in what are sometimes called hidden Markov models. We're ignoring all of that. We're approximating it away. That's what makes it powerful. And there are things you can do like you could have the um size of the Gaussian noise. The characteristics is called of the vener process. Those could change across your space. You could have there be a more sophisticated ways of changing this some uh underlying vector of directions and how it varies across the space. Your step that you're going to take and then add a little bit of noise to how that step, the direction, the size of that step, how it varies across the space. You can vary all these kinds of things. What makes it again so powerful as a tool to analyze human societies is precisely the fact that you don't need to know all these earlier um values of my state. To figure out where I'm going to go next, I don't need to know where was I yesterday, the day before, the day before, and the day before that. In contrast, things like what are called an auto regressive model or standard nonlinear time series, you need to know all those values as well, and that vastly complicates the entire analysis. So that was, that's what makes it a good tool. The underlying Philosophy. Is the following. There's a vast amount of work in social sciences, and I've done it as well, cause I used to do a bunch of work in game theory, but, uh, game theory as a field is kind of broken, so you can only do so much there. Um, BUT, uh, the vast majority of social scientists, including game theory, they are based on introspection. We all sort of think about, oh, Ricardo, what do you think are the important issues concerning um how a society will evolve? Oh, David, if I were a society, I would be paying attention to, um, you know, I think people care about the rate of inflation. OK, that sounds good. Let's put that in as a variable. And when we're talking about just person to person interactions, social networks, what do we think is important? Well, um, uh, my buddy Ricardo. Do I think that Ricardo, um, is somebody who was good to me last week? Yeah, he, you know, he did, he did me some good, good, good turn last week, so I'm feeling well to Ricardo right now, so I'm more likely to give him a little bit of slacker, you know, whatever it is that I can give. We do this all based on introspection. We don't do that for any of the other sciences. If you consider something like physics or chemistry or biology, it's crucially all based on observation. You let nature tell you, the scientists, what is really going on. Um, WHEN I, uh, in another previous career, I was an aerospace, um, an aeronautics professor at Stanford. One thing that I used to say that was, I found to be so important about doing engineering as opposed to just pure theory. Is that that provided a way for nature to tell me what the truth was rather than me to just introspect and think how it should go. Nature could take my brilliant, absolutely beautiful theoretical work, and then when it doesn't fly, nature turns around and says, so nature can turn around, I'm a human too. So nature can turn around. And they can kick me in the ass, saying, David, you thought you were so smart, you thought you had it figured out how the world really is. You're wrong. Your airplane just fell out of the sky. Go back to the drawing board, think about it again. This whole body of work where we just had these 50 variables, that far more than we would think about in terms of introspection. These are the ones that have been found working out in the archaeological field. In this case, by looking at these flow patterns, these stochastic processes, going through these spaces of 50 variables, we are allowing nature to tell us. What is going on. We can sit here and think, oh, I would expect that once um my society grows to be the following size, then it might just keep on growing. It doesn't have anything to do with the computational power. No. Nature tells us, that's what's so important about this new body of work for analyzing history. It's especially very, very different from the standard approach in what's called history or historiography, which is all based upon storytelling, it's narratives. It's the last refuge. Of people from the 19th century who did not want to introduce numbers and mathematical reasoning into our understanding of the way the world works. That's in many ways, unfortunately, the profession of history. And this is just blowing that out of the water, just walk, stepping around, all of that to let nature tell us what's going on.

17:40 Ricardo Lopes: And in what ways is history a stochastic process?

17:45 David Wolpert: Um, BECAUSE the evolution is noisy. As you and I were discussing, um, at the very beginning of this interview, we don't know. We can make some predictions, and they might be likely to be true, but we don't know what the state of the world is going to be. Is Ukraine going to, well, to just sort of speak politically, is Ukraine going to face reality and actually form a peace treaty where Putin will get some of the territory but not have one, or is Ukraine going to do a battle to the death? What's going to be happening to the Chinese economy right now? It's having a lot of difficulty. Is that going to continue and is it going to actually impede the possibilities of China, um, slowly taking over the South China Sea? Um, IS the United States going to invade Greenland and the Panama Canal? What's going to happen to NATO? We don't know. These are all random, so that's the stochasticity. um, IN the stock market, what's going to happen to the world's stock markets in, um, in Japan, the Nikkei, in the US, in Europe, what's going to be happening? These are all social systems. These are all part of history. They will go into the history books that are written about 2025, but it's random, so it is a stochastic process. That's the key. Everything in the real world is a sarcastic process because we cannot model things down at the level of atoms and molecules, which is what which we would need to do to actually be able to view it, model it as a deterministic process.

19:29 Ricardo Lopes: So as an example in the paper, you talk about jumps in the socio-political complexity of politics. What can we learn about them through this framework?

19:40 David Wolpert: Um, PRECISELY because of the, um, I gave as an example, maybe it was, I should have waited until now, these two thresholds, if you look at If you very roughly decompose the um the dynamics of the systems in the space of 50 variables down to two separate variables. One, as I was mentioning, measuring the computational power of a society, and one, just its size, how many people are in your capital city, that kind of thing. So here is size, here is computational power, trying to get it right. Um, AND what you would see, there are very clear thresholds. It's a sawtooth pattern. It does this. It's not even a sine wave, it's very sharp, and it's true for all societies. And so what we have learned, you, I could now, we could discover a new society somewhere. Figure out what is its population according to um this measure, and what is its computational power according to this measure, we would see, oh, it is right now in between this jagged tooth and pointing down and this jagged tooth pointing up. So we could then predict. Not exactly, but to high accuracy. Its future is going to be that it will keep on growing its um computational power until it hits that spot, and then it will be growing in size again. We can now do these things looking into the past. It's amazing. Nobody thought there would be these kinds of rules. It's almost like Isaac Asimov's foundation series, but it's all driven purely by the data rather than by mathematics.

21:29 Ricardo Lopes: So just a few minutes ago, you talked a little bit about narrative approaches in the history. Is a stochastic process framework supposed to replace them or to complement them?

21:43 David Wolpert: Compliment, I guess, um, a more. Properly speaking, it's a very strange thing, the sociology of the sciences of archaeology, anthropology. And history. Mostly, I guess I'm talking about the West, um. So what happened was, everything was very, very quantitative, numerical, um, uh, economics, um, a whole bunch of the famous, um, economists from the 19th century were also famous, um, uh, physicists, for example. And then there was also Darwinism and the idea that everything is a competition. All of that is completely accepted. It's true. But there was a political process in the West. It was called social Darwinism, eugenics. People misinterpreted, people who were not scientists, politicians, they had other purposes in mind of what they wanted to do rather than just uncover the truth. They um took these insights. And misunderstood them, misused them just like misinformation. It's, it's an age-old kind of phenomenon to try to argue certain races are intrinsically less advanced, they are all deficient compared to others. Of course, their race, of course, was the best one, and their country was the top, and so on and so forth. This was social Darwinism, this was eugenics, people going so far as to say that you should. Um, GIVE rights for having children based upon the intelligence of the adult. There was then a very, very strong reaction against this. That reaction, it was very vicious and it was an overreaction. There's similar things in the field of biology. There's always overactions and then overreactions. It's the way these fields very often tend to go, the ones that have large aspects to them that are not just mathematics-based. Um, uh, BUT it was particularly pronounced. Anthropology and archaeology have started to recover. You can now talk about equations and so on in those fields for the most part. History still has not. It's now you will. There Jokes made about how if you were to go to the annual meeting of the American Historical Association. And propose a session on numerical approaches to world history. There's no way that anybody would even consider having such a special session. It's just, it's dogmatism, it's theology. Um, THAT this is not what history is, and that Thesis is actually very often. Presented associated with some. Almost moral. Reaction that's to try to introduce numbers into these kinds of analysis of human nature is immoral, it's evil. It almost has that kind of a feel. Nobody says that. But it, and they might not, nobody that might not even be conscious, but that is how strong the reaction can be.

25:22 Ricardo Lopes: So I have one last question then talking about the future now and not so much about the past. Would this sort of framework, a stochastic process framework also allow us to make accurate predictions about the future and if so, what kinds of predictions?

25:42 David Wolpert: Of course, there's a cliche. Prediction is hard, especially about the future. Um, SO, Yes and no. We are starting this we as a very broad group are starting to Bring these analyses up to involving more recent data sets. The kinds of stuff I was presented before Seat, it's some successor, which is called Equinox or many other data sets is what's called the Human Atlas. Of um I forget what it is, a human atlas or something or other, um, the these data sets mostly deal with the deep past. When you come up closer to the present, closer to the modern day, the past 500 years, Your problem is no longer not enough data, it's far too much. Rather than only being able to um uh record 50 variables, because that's all that's left in the ground that we can actually look at, now there are millions, trillions, you know, billions of variables that you could actually consider. So, to apply these tools to more modern situations, there is some work, a lot of it coming out of the Santa Fe Institute, people like um Hidalgo and Bernardo Uber and Bernardo um. Uh, I'm blanking on his name. It'll come to me. Um, THESE are kinds of people, Housman, Ricardo Hausman, um, these people are doing certain kinds of analysis along these lines. Um, THERE'S also work being done by a fellow called Peter Churchn. Um, THERE'S also me and some of my other collaborators are starting to try to, um, analyze more recent data sets. It's a much more Challenging exercise and analyzing the past because there's too much data now. There are other issues as well. In a certain sense, history is speeding up. So when we were looking at the past, What was going to be 1000 years of changing of one particular society in the past can now happen in like a year, 5 years at most. It's very, you know, depends who knows how to precisely measure it, but it's very, very fast because population densities are so high and so on, and this has to do with scaling behavior, what are called nonlinear, um, scaling properties of human cities and so on, and there. There's a huge amount of work by people like Luis Bettencourt and Jeff West, and Chris Campus, other people associated with the Santa Fe Institute, and also many sociologists who were um preceded them to actually start analyzing all of these data sets about super linear scaling. What that means is that in a very real sense, the speed of historical changes is faster in large dense cities than it is. In lower density ones, populations have been going up, so that's speeding things up. Technologies are changing very, very quickly. All of these things mean that what we could worry about for how accurate the predictions were for 10 or 2 1000 years when looking at the past. Now might be the only kinds of accuracies we could even hope to come to for looking into the future for 5 years, except now that we have so many more variables, so we're not even sure how to do it. So, long, long-winded answer, but the end of it is, yes, these different approaches that I've been talking about. Can be used to make predictions, geopolitical, um, economic predictions. We are sociopolitical predictions. We are starting to think about how to do this, but it is far more challenging, and those predictions when we do have them, are going to be far um less certain. They're going to be less crisp than the ones that we did about the past.

30:07 Ricardo Lopes: Great. So, doctor W purchase just before we go, would you like to tell people if there are any places on the internet where, where they can find your work?

30:17 David Wolpert: Um, THE easiest stuff, I guess, is my scholar page. You can also, um, I've got a website, um, I think it's called this, um. I think that's it. Um, MY own, uh, so you can find a whole bunch of my stuff there, but mostly in terms of this kind of work, it's on my Google scholar. Um, I just, the thing is, I work in a huge number of fields. I work in what we've been talking about today. I also work in computer science theory. I work in a field called stochastic thermodynamics. Analyzing the stochastic thermodynamics of computers like the one in your head right now, you call it your brain, or the one that we're, you know, the other very similar computers that we're right now conversing over. Um, I've got other bodies of work where I'm looking at foundations of math. I have, I work on very, very many things. I'm still doing a little bit of machine learning way back like 40 or 50 years ago. No, no, not that long. Um, 40 years ago, I was actually one of the original people doing machine learning. Before I frankly got bored of it. Um, I got bored of it and I was also actually kicked out because I came up with things called the no free lunch theorems. And people did not like those, um, it was pretty vicious. Um, BUT any case, so I've been working on all these kinds of things. So what we're talking about today, you can find it on my scholar, Google Scholar, but it's it's um buried in with all these other kinds of papers as well.

31:59 Ricardo Lopes: Great. So thank you so much for taking the time to come on the show. It's been a pleasure to talk with you.

32:05 David Wolpert: Great. It's been, uh, uh, really fun to, uh, interact, to have the conversation. Thank you very much.

32:13 Ricardo Lopes: Hi guys, thank you for watching this interview until the end. If you liked it, please share it, leave a like and hit the subscription button. The show is brought to you by Nights Learning and Development done differently, check their website at Nights.com and also please consider supporting the show on Patreon or PayPal. I would also like to give a huge thank you to my main patrons and PayPal supporters Pergo Larsson, Jerry Mullern, Fredrik Sundo, Bernard Seyches Olaf, Alexandam Castle, Matthew Whitting Berarna Wolf, Tim Hollis, Erika Lenny, John Connors, Philip Fors Connolly. Then the Matter Robert Windegaruyasi Zu Mark Neevs called Holbrookfield governor Michael Stormir, Samuel Andre, Francis Forti Agnun Sergoro and Hal Herzognun Macha Joan Labrant John Jasent and Samuel Corriere, Heinz, Mark Smith, Jore, Tom Hummel, Sardus France David Sloan Wilson, asilla dearraujuru and roach Diego Londono Correa. Yannick Punterrumani Charlotte blinikolbar Adamhn Pavlostaevsky nale back medicine, Gary Galman Sam of Zallidrianei Poltonin John Barboza, Julian Price, Edward Hall Edin Bronner, Douglas Fry, Franco Bartolotti Gabrielon Corteseus Slelitsky, Scott Zacharyishim Duffyani Smith Jen Wieman. Daniel Friedman, William Buckner, Paul Georgianeau, Luke Lovai Giorgio Theophanous, Chris Williamson, Peter Vozin, David Williams, the Augusta, Anton Eriksson, Charles Murray, Alex Shaw, Marie Martinez, Coralli Chevalier, bungalow atheists, Larry D. Lee Junior, old Erringbo. Sterry Michael Bailey, then Sperber, Robert Grayigoren, Jeff McMann, Jake Zu, Barnabas radix, Mark Campbell, Thomas Dovner, Luke Neeson, Chris Storry, Kimberly Johnson, Benjamin Gilbert, Jessica Nowicki, Linda Brandon, Nicholas Carlsson, Ismael Bensleyman. George Eoriatis, Valentin Steinman, Perkrolis, Kate van Goller, Alexander Hubbert, Liam Dunaway, BR Masoud Ali Mohammadi, Perpendicular John Nertner, Ursulauddinov, Gregory Hastings, David Pinsoff Sean Nelson, Mike Levin, and Jos Net. A special thanks to my producers. These are Webb, Jim, Frank Lucas Steffinik, Tom Venneden, Bernard Curtis Dixon, Benedict Muller, Thomas Trumbull, Catherine and Patrick Tobin, Gian Carlo Montenegroal Ni Cortiz and Nick Golden, and to my executive producers, Matthew Levender, Sergio Quadrian, Bogdan Kanivets, and Rosie. Thank you for all.