#1005 Joshua May - Neuroethics: Agency in the Age of Brain Science

00:00 Ricardo Lopes: Hello, everybody. Welcome to a new episode of the Center. I'm your host as always Ricardo Lob. And today I'm joined by Dr Joshua May. He's Professor of Philosophy and Psychology at the University of Alabama at Birmingham. And today we're talking about his latest book, Neuroethics Agency in the Age of Brain Science. So Dr May welcome to the show. It's a pleasure to everyone. Thanks for having me. So, in the book, you go through actually several different uh topics that I've explored on the show with other people as well. But before we get into the specific topics just to introduce what neuroethics is to the audience. So, could you tell us a little bit about this? I'm not sure if we should call it a specific sub discipline or something like that. But what is neuroethics? And what kinds of questions does it deal with?

00:56 Joshua May: Sure. Yeah, neuroethics is a fairly new discipline. It's really only about, let's see, 2025 years old is around the turn of the century that it got its own label and, and got pretty clearly defined. But the idea is, it's sort of evolved out of bioethics. It's kind of like bioethics for the brain. But it's not just that it's also about how neuroscience can tell us about some classic old questions and ethics. So it's usually thought of as well. We can think of neuroethics is partly just how do we do neuroscience ethically. That's kind of like bioethics for the brain. But then there's also the idea that what can neuroscience tell us about ethics. So there's this, this nice phrase, the neuroscience of ethics and the ethics of neuroscience. And and it includes both of those which, which is really great. It means it's a very wide ranging field and it's one that is, is both trying to sort of ethically police neuroscience, but also to, to learn from it. And that's one of the things that's so exciting for me is that we get to ask both of those kinds of questions in neuroethics. So we can ask questions like when is it ethical to manipulate people's brains? There's a lot of new technologies coming out. Uh The are promising to intervene on people's brains for, for medical purposes, but also really just for the average consumer. Uh Elon Musk has this, this company neuralink and there, there are many others that are coming out with, with a lot of um brain computer interfaces. Um But then there's also questions like, well, what is neuroscience telling us about how moral judgment works? Um OR about about addiction or mental illness? Uh Things like, like neurodiversity and, and we're, we're learning sort of back and forth going from thinking about the ethics and thinking about what the neuroscience tells us about ethics. So I try to tackle at least some of those questions in the book. I couldn't cover them all. I mean, it's, it's already, I think overly ambitious trying to cover a wide range of topics. Um, A lot that I left out. Um BUT, but I, I go from talking about free will to addiction to brain technologies, brain interventions, uh and the use of neuroscience in the courtroom as well to, to potentially um you know, find people guilty or innocent based on their brain activity.

03:06 Ricardo Lopes: Uh So something that you mentioned there, I, I think that is probably very contentious. So for example, when it comes to uh saying that neuroscience can inform ethics, uh I mean, the ways that science can inform a sort of a normative field, let's say if that science can tell us what we should do, what we ought to do, that is the kinds of questions that ethics explores. I mean, I would imagine that, but uh that would be among moral philosophers and ethicists, a very contentious question, right? Because that it is not at all a straight, it doesn't have a at all a straightforward answer to it. Right. Right.

03:57 Joshua May: Yeah. And there's this, this old is art gap that, you know, David Hume famously introduced us to. He said, and you can't just go from, here's how things are, uh, to how, here's how things ought to be. And that question I think is still not exactly resolved, but most people who are working in neuroethics just concede that that is a gap and that we need to bridge it with various principles. So we can't just go from, you know, here's how the brain works to, you know, here's how free will and more responsibility and moral judgment, you know, ought to work. But we can bridge that with various principles and we could say, ok, well, if you know, this is how things are working in the brain, um, maybe it can tell us something about ethics. So I think that you have to have some of those assumptions in there and it's why we do need, um, ethics involved when we're trying to draw those conclusions. We can't just go from the science to the ethics without some bridging kinds of principles.

04:53 Ricardo Lopes: Mhm. Yes. And I guess that by exploring some of the topics we're going to talk about here today, people will also get the notion of how the two way street between science or neuroscience specifically in this case and ethics work. So, uh, let's talk a little bit about free will here. I know that from a moral perspective, free will connects with questions regarding moral responsibility, agency, stuff like that. But I mean, in what ways do you think that neuroscience as a scientific discipline can be informative here. I mean, how would people tackle neuros uh uh free will from a neuroscientific perspective? Because I mean, I, I posed this question actually to um last year to Kevin Mitchell who is a, a neuroscientist himself. Uh I asked him, uh so why did you get into a question that originally or traditionally has been tackled by philosophers and not scientists? And since you are a philosopher, I guess that I want to ask you the same thing. But from the other perspective, how can science help you, let's say,

06:12 Joshua May: yeah, talk about uh controversial. That's one of the the oldest most controversial questions in philosophy, right? Free will. But, but I think it is, you're right, it's a good way of thinking about it. There's a two way street and for a long time, philosophers have talked about free will without really thinking much about the science. And they haven't really been able to, to know much about what's going on in the brain because, you know, neuroscience is, is not that old of a discipline uh either. And so you see some philosophers who aren't paying much attention to the, to the science, but also some scientists who maybe aren't paying much attention to the philosophy. So I think the best work in this area brings the two together and as philosophers, we should be humble and, and pay attention to what's going on in science. But the same goes in reverse So I think for a long time, you know, philosophers have been worried about free will by looking at physics. They say, well, maybe uh physics shows that the, the world is determined. And so that means that all of our actions are determined and that seems like it might threaten free will if, you know, I have no real choice in the matter of what I'm going to do. Maybe it's all predetermined by the laws of physics and in the past. But the, the more modern threat that's coming from neuroscience is a little bit different. The idea is that we're not just learning that our actions are determined by physical forces, but that they might be largely determined by unconscious forces. And that's something that it does seem like we're getting a bit of consensus out of neuroscience and cognitive science more generally that a lot of our decisions are driven by unconscious forces. Uh Freud was at least roughly right about that. Um There are a lot of unconscious factors now, they may not actually deal with, you know, um repression and, and a lot of uh problems with, with our parents and, and the upbringing, but that's part of it and part of it is just, you know, incidental factors that, that influence us in the present moment that we don't understand uh things like biases. Um OR if you're just being particularly irritable, um there's something a foul smell in the room or something that could be influencing your decisions and you might not realize that or think that it is. So that is certainly a, it's a consensus now that a lot of our decisions are influenced by unconscious forces in the brain. Now, for many philosophers and scientists that seems like it threatens free will because we tend to think that our free choices come from our conscious mind. It has to be that I consciously choose, you know, to say certain words to people or, or make certain decisions for that to be free. And that does pose a major challenge. But we have to do that both that work from philosophy and science. You have to look at science say, what is it really showing, you know, how much of our decisions are influenced by unconscious forces. But then also look at the philosophy and say, well, does that really show that free will doesn't exist? What do we mean by free will? And I think that actually the science can help us understand that as well. We can do studies to try to figure out how do people understand free will, you know, what does it mean? And my own view is that I think once we get all that settled and on the table that we don't necessarily have a threat to free will, we might need to revise our conception of free will. Maybe it's a little different than we thought it was, but it doesn't necessarily mean that we have to reject it.

09:22 Ricardo Lopes: And I guess that there's also two different sets of interesting questions here. One is whether free will exists is a real thing, whatever it might be or not. And the other is whether free will exists or not, people have reasons for attributing more or less free will to other people in different circumstances. Like for example, when they want to blame them or praise them, they usually attribute more freedom to their actions. But uh if they think that, I mean, they are, for example, the victims of their circumstances or something like that, something bad happens to them that they had no control over, perhaps they are more, are less willing to attribute freedom to their actions. Right? So I, I mean, those are two different sets of, I would say interesting and important questions here, right?

10:21 Joshua May: Yeah, that's a good distinction to draw partly because philosophers for a long time only thought about either free will exists or it doesn't. And they haven't thought about it coming in degrees. And I think that's a great development in the philosophy side. And many philosophers today would say, well, no freedom. And so responsibility and blame can come in degrees. And I think that's right. And so we could even see the science as suggesting that maybe we have less freedom than we thought, but we don't completely lack it. Now. I think that that's, that's right that we're, we're learning that there's a lot that influences our, our decisions that we weren't quite fully aware of and that might actually mitigate blame. I said, well, it's kind of like when you find out that somebody, um, you know, stepped on your toe accidentally, um, or, you know, you think that somebody's been really not a very good friend lately but you find out that, well, it's because they're going through a divorce. Right. Or they've got some ailing parents or they just haven't been sleeping because they've got a newborn. And so you go, oh, ok. And you said, well, I guess I'll blame them a little bit less, but you don't treat them like there's some machine that has no free will. You just say, well, you know, I'm gonna blame them less or hold them less accountable. And I think that happens all the time through ordinary life. So maybe it's not such a surprise if we find out that the science is suggesting that, that we, we need to do some more of that. We need to say, ok, people have a lot of different forces that are influencing them sometimes. That means they're a little bit less responsible, but we're still responsible in general and we're still the kinds of creatures who can be held accountable for our actions and make free choices, even if it's true that we have less freedom than, than we thought.

11:55 Ricardo Lopes: So, at a certain point there, when I asked you, first about free will. You mentioned the fact that we now know through science, neuroscience, psychology and so on, uh that we have many different unconscious mechanisms operating in our minds that of course influence, at least to some extent our behavior, our decision making and so on. Do you think that consciousness is strictly necessary for free will to exist? Or do you think that perhaps free will would still be possible even in the absence of consciousness?

12:35 Joshua May: Hm. Yeah, that's a great question because my own view is that consciousness isn't always necessary for freedom, but it would be a little bit different if you had no conscious choice in the matter at all. And, and that would be, you know, this idea of, of Pien very, very mouthful of a, of a label there. But the idea that our conscious choices are pi phenomena or, or they just come along for the ride, they're just kind of byproduct's if that really was the case that, that our conscious decisions never really played a role in, in our actions that I think could be potentially a threat to free will. But I think what the science is actually showing us is that those conscious choices just play a bit less of a role in our minds. And so to me, that doesn't necessarily mean we, we lack free will. Uh PARTLY because I think that we are these complex agents, we have both conscious and unconscious forces. And I think that we shouldn't just identify ourselves with one or the other. We tend to want to identify ourselves with the conscious part. So, you know, I'm the one who consciously decides and all this unconscious, you know, biases and things that I'm not aware of. Not part of me. Uh I think that's a mistake. I, I think it's true that we tend to identify ourselves more with the conscious parts, but this is all part of me. You know, it's not part of anybody else. Uh You know, the, the unconscious biases and thoughts I have are, are, are not part of you. They're part of me. And the analogy I like to draw is with a corporation or really a large agency where you have places like Apple has a CEO and we often tend to identify the company with the CEO. We said, well, you know, Apple kind of is primarily driven by Tim Cook the CEO, but of course, Apple is a much bigger entity than that. It's not just the CEO, it's not just the part that's playing that sort of executive function role, as we often say in neuroscience, it's part of it and it's important part, but it's not the only part, there are a lot of other features of Apple, like the, you know, level level managers, all the workers and they all actually direct the company and move it in different ways. Now, Tim Cook might not be aware of a lot of that activity. Um But it's part of what Apple does, it's part of its agency. It's part of how it acts and makes choices in the world. And if you have no CEO whatsoever, you might have a completely disorganized organization and feel like there's not really any single entity here. So maybe a CEO can help to kind of unify things and provide some sort of direction, but it's not the whole agency. So I think similarly, you know, our conscious minds probably have to play some role in unifying our, our, our otherwise uh disorganized mental states, all the kinds of automatic thoughts and desires that pop up all the time. But all those things are part of who we are and we, we can't distance ourselves too much from them. And we, we need to maybe revise our conception of our, of ourselves and our freedom to say a lot of that is part of our ourselves and our choices. Um EVEN if uh the conscious parts must be involved as well.

15:37 Ricardo Lopes: Uh And I guess that also related to consciousness, there's the fact that even if uh lots or even most of what happens of or what goes around in our minds is uh unconscious or plays out at an unconscious level. I mean, at the very least, uh I would say that we have veto power over what comes out of those mechanisms in terms of how they influence our behavior. Right. I mean, a, at the very least before we do something we can decide not to do it or go through some other course of action.

16:21 Joshua May: Mm. Right. That might be enough for, for free will. Yeah. The, the neuroscientist who is responsible for a lot of this early work. Benjamin Labette actually did think that we had free will just because of that. He thought we had, well, he called it free won't, uh, that, you know, we can decide not to do things. You know, I'm not so sure that that's enough though for, for free will. I think that we, we really don't always have that possibility to, to veto the actions. I think that the science does suggest that we often are kind of on autopilot a lot. There's actually, uh, one philosopher who's been really pointing this out a lot lately is, uh, Shandra Sri Pata. He's at Michigan and he said we need to really think about the default state of the human mind as basically more automatic. It's kind of like it's, it's like a car where if, you know, you let off the, the brake, it just tends to move forward. Um, EXCEPT apparently on Teslas is different, but, you know, on, on most cars you let off the brake, it just sort of goes out of it. And I think that that's kind of how the mind works as well. And cer certainly, I think Sru Pada does as well. That there are all these different kinds of thoughts that just bubble up automatically, these different desires and impulses that, that come on automatically. And we sometimes have the ability to, to veto and control. But if a lot of it's unconscious and automatic, then we might not be aware of it. And so it's hard to veto something that you're not aware of. So I think if we said, uh we have free will as long as we can veto every possible automatic urge or thought that comes up, then I think the science would, would suggest that we, we don't necessarily have that. Um, BUT I do think that we have some role for consciously influencing our automatic thoughts and having that kind of cognitive control sometimes to maybe direct those urges and thoughts or to, to tamp them down and not let them influence behavior. But a lot of times we're just kind of on autopilots and, you know, I may just automatically have some thoughts and, and let them fly at a party. Um, AND some of them might be, you know, a little bit insensitive, but I can't say, well, I didn't consciously choose to, to veto that insensitive remark. Um, OR I didn't consciously choose to generate that insensitive remark. It just, it just happened, it was automatic. Um I think rightly, a lot of still hold each other accountable for that. We said, well, you know, you, but it was, you wasn't anybody else, uh who, who said that insensitive remark and, you know, it's part of, it represents part of who you are, uh represents part of the kinds of automatic values and, and um desires that you have going on in your mind. And so I think that we still have to, to some degree identify with those aspects of, of uh our agency. It, it'd be kind of like if, you know, Tim Cook says, you know, I didn't realize, you know, there's all this like spying going on with, you know, all of our devices, you know, listening to you through your iphones, I didn't realize it's just that the company, all these managers and people, he still is going to have to be, he and the company is going to be held accountable for that. It's part of what Apple did if they're, if they're listening to us on our iphones, uh even if the CEO wasn't fully aware of that, it kind of happened on his watch, so to speak.

19:24 Ricardo Lopes: Uh WHEN it comes to this kind of topic, one of the examples and uh since you talk about being on auto autopilot mode there or something like that, uh One of the examples that I love the most, even though of course, compared to other kinds of bigger decisions that we have to make in our lives, it's relatively small. But uh is the one of driving a car. I mean, because uh for example, if I go to a new city, uh, in my car and I don't know the city, I don't know the streets, the roads, I mean, I usually need to pay more attention and so the attention really brings a con anxiousness to the forefront because I really have to pay lots of attention to all of what I'm doing. But if I'm driving on the road I've already driven on 500 times. I'm probably on autopilot. Right. Because I really don't need to pay much attention at all to what I'm doing. And, uh, I mean, we, we drive relatively well, even without paying much attention if we are experienced drivers at least.

20:34 Joshua May: Right. Yeah. Yeah. It's such a familiar, you know, experience that it's almost scary sometimes, you know, you're driving and then you arrive home and you realize, like, I can't remember any of what I just did. Uh, YOUR mind was wandering elsewhere and, you know, you just did all, but it wasn't, it wasn't like you weren't making choices. I mean, and responding even to the environment, you know, they had their stoplights and their pedestrians and so your brain is still processing all of that and responding to it. It's just that your conscious mind wasn't really attending to it. And so I think it's a great example of, of choices we still make that we still make freely even though they're not fully conscious. And so if somebody, you know, is, is on autopilot while they're driving and then they make a mistake. Um, THAT'S still something that they've done, still something they could be held accountable for. And I do think that the, the neuroscience is suggesting that more of our behavior is like this, then we tend to think, but that, that's brings us to the, so we have to do the science there and figure out what is it really telling us? Um, AND I think it's not telling us that that everything's on autopilot. So some scientists do believe that that kind of epiphenomenal is that it's just never involved that our conscious choices, you know, seem like they're involved, but they're really just kind of floating along uh like the steam that comes out of uh you know, a train engine, it's not really actually causing the train to move, it's just a by product. I don't think it's showing us that. So it's showing us that well, our conscious decisions don't play quite as much of a role as we might have thought. But then we can now get a switch in philosophy mode and say, well, does that show that we don't have free will? Uh DEPENDS on what we, we mean by that, I suppose. And, and that itself is, is controversial, but the basic idea of free will and, and there have been studies on this suggesting that this is a very ordinary conception is just the ability to make choices among genuine options in light of reasons, you know, to, to, to make choices in, respond to the reasons that you, you've got. And I think that that can happen even when it's pretty automatic and unconscious. So if you're on autopilot driving, you're not consciously aware of every decision that that's being made. But you, your brain is making choices responding to reasons. You know, if the light is red, uh then you, you push on the brake and you stop the car and that is a choice, responding to reasons among different kinds of options. You, you could have punched it and blown through the light right, or not stopped and that I think can be enough for free will. Um As long as we've got, you know, a complex that's unified in the right sorts of ways that has a certain identity. It's not just this disorganized kinds of urges and automatic thoughts. Uh BUT much of our mental lives may be on autopilot and yet still fit that basic definition of free will that, that we can make, we can respond to reasons uh in our environment. And that is a pretty, it's not completely uncontroversial. Um BUT it's a very popular view about what free will means among Phil first, but apparently also the general public, there have been, you know, psychology studies on this asking people, you know, what are the factors that are relevant to free will giving them hypothetical different kinds of scenarios to see if they think someone acted freely or not. And so there are a lot of scientists who have a certain conception of free will, they think, well, it has to be conscious decision all the time that can like defy the laws of nature and defy your own brain chemistry. Certainly we don't have that power. Um The neuroscience shows we don't, we don't have that, uh we are, you know, uh physical creatures with, with these electrochemical brains. And if we're going to make choices, it has to work through that and a lot of it's unconscious, but we don't necessarily have to define free will that way. Uh A lot of scientists do and I know he had a Robert Souls on who's, who's got a recent book like like this. And he, you know, he thinks, well, this, this means we don't have free will. Well, a certain conception of free will, I suppose. But the analogy I like is with um you know, witches versus solidity. So when we find out that there's no magic or what have you, we say, well, then that means there are no witches because to, to be a witch is to have certain magical powers. But then science revealed us to us also that um solid objects are mostly empty space. A bowling ball is mostly empty space. And yet we don't say, ah, so there's no such thing as solidity, we say, oh, well, solidity is a little bit different than we thought it was. And I think that we can and should say the same thing about free will that we often make this mistake of saying, well, we have to always make conscious choices if it's going to be free. But the more we learn about what kinds of creatures and agents we are, we can start saying, ok, well, maybe that's not really what we are. We are these complex beings that are full of conscious and unconscious choices that play a role in our decision making.

25:20 Ricardo Lopes: So, in sort of a related topic in the book, you also explore the issue of brain manipulation. So before we get into specific cases of brain brain manipulation, and you go through for example, ethical questions associated with medicine. Uh WHAT is brain manipulation? What counts as brain manipulation? Exactly?

25:46 Joshua May: Yeah. When we think brain manipulation, we think coercion, you know, forcing someone to, to behave or think a certain way. And uh I was thinking of it more broadly as it could be even also the uses of brain interventions to treat um medical conditions. So there are all kinds of ways we can manipulate our brains and they, they are classic ones like drugs. Um Ssris or other antidepressants are ways of manipulating our brains. And back in the nineties, there are a lot of ethicists worried about that. They thought, oh no. Now we're starting to change our very personality and that could be kind of ethically problematic, but we've become a lot more comfortable with that so. Well. Ok. Sure. I'm taking a pill that, that alters the serotonin in my brain. But that's something that I can freely choose to do and it may alter my personality a bit. But, but that's something still that I've chosen to do freely. Now, a lot of ethicists are getting more concerned about even more direct brain manipulation. Uh THINGS like brain stimulation, uh It could be invasive or noninvasive or other kinds of drugs like psychedelics. So there are a lot of uh there's a lot of research now about psychedelics suggesting that we could be used as a, as a form of uh treatment for medical conditions, uh possibly also a form of, of enhancement. But a lot of people are concerned now that that might be an unethical way of manipulating our brains. It certainly does change the brain when you, when you have psychedelic experiences. And there are all these kinds of ethical questions that, that are raised when we start to mess with um or manipulate the brain because it's not really just like having an operation on your arm or your leg. Uh Now we're talking about what I like to think of as the seed of the self. You know, this is where our free will and our personality and our values is is roughly housed in the brain and our nervous system. And so it does raise this question of, you know, are we uh impairing someone's autonomy by manipulating their brain or just having a relevant brain intervention? Um OR can we, can we freely choose to change our own personality and identity? Uh OR are we risking too much harm by manipulating the brain? Especially when we don't really know a lot about how it actually works just yet. Uh We're still quite far from that actually, we're, we're still learning a lot about how the brain works. And there's, it's a big mystery how all this firing of neurons gives rise to compliment uh complicated psychological states, like emotions and decisions and all of that. And so there might be a real concern about whether, you know, neuroscience is ready for, you know, the big stage of, of manipulating people's brains to try to treat even very serious kinds of medical conditions.

28:33 Ricardo Lopes: It's very important that you clarify that because I mean, I would imagine that most people, when they hear the term brain manipulation, they would think mostly of surgery, perhaps deep brain stimulation. Some people also think about that but not necessarily about uh drugs in general, like for example, antidepressants or anxiety medication. But here you are including anything that can have any sort of uh biochemical electrochemical effect on the brain as well. Right.

29:10 Joshua May: Yes. Yeah. And I think that's important because we shouldn't be concerned about, you know, brain stimulation just because it's a bit new and involves even more direct manipulation of the brain. That doesn't seem to be a morally relevant kind of distinction. There's still all the same moral questions arise when we're talking about things like uh psychedelics or antidepressants, which you know, many, many more people are taking. You know, there are a lot of patients who have been doing things like deep brain stimulation for Parkinson's and other neurological conditions. But there are many, many more people who are taking drugs for mental disorders, things like depression and so forth. So we need to, I think raise the same kinds of questions there. But of course, what's new and, and a little bit more kind of the Wild West for ethics is direct brain manipulation. And as you say, that could be done through things like deep brain stimulation where you have to actually have surgery. They, they cut open the skull to implant very thin electrodes deep in the brain to, to treat neurological conditions like Parkinson's that involve degeneration of the neurons deep in the brain. And, and that actually has been quite successful for, for certain patients who have movement disorders like Parkinson's or essential tremor, but it's being expanded now to a lot of other conditions, psychiatric conditions like treatment resistant depression, um anorexia, nervosa, uh O CD. And so there are concerns I think about, you know, are, are we using this uh appropriately for all of these different ranges of conditions? You know, is it safe? Is it really going to benefit people. But those are the same kinds of questions that we can ask about uh antidepressants. And there have been big debates about whether antidepressants are actually doing much for people and whether they're very effective and whether they have risks that we might not be so aware of. And uh it's something that has affected me personally because I've taken drugs before that have been on the market for decades. And then we find out later that they, you know, increase the risk of heart disease. Um I, I get migraines and so I, I've had a drug that got pulled for that. Ok. Well, uh surprising because that drug was out for, for decades and then we find out that it's causing some harm. So I think we always have to worry about that with any kind of medical intervention, but especially with brain interventions, we have to figure out uh is it really benefiting patients and how much and then weigh that against the risks and say, well, is it actually risking more harm than good? So we always have to do that risk benefit analysis. And when it comes to manipulating the brain, we also have to factor in its potential risks. Uh THINGS like changes to a person's identity and personality. Um And those are, those are relevant things to weigh into that calculation.

32:02 Ricardo Lopes: Uh You know, when earlier I mentioned surgery, I was also thinking about probably even more invasive things like lobotomies because you know, I am from Portugal and Agus Muniz who got the Nobel Prize because he was one of the pioneers of uh lobotomy procedures. I mean, we know now that people who got that kind of intervention back in the fifties, sixties and so on due to epilepsy, for example, to cure epilepsy, the epilepsy got cured, but then they didn't uh end up in a very good psychological state. They lost some important, I mean, the decision uh important mechanisms related to decision making, emotion processing and all of that. So I guess that would be uh also a very good example of brain manipulation in the extreme.

33:02 Joshua May: Right. Right. Yeah. And we look back at the days of lobotomies as a kind of dark period of, of neurology. But it's important as you say, I mean, this was Nobel Prize winning work and it did actually, you know, work in some cases. The problem really was when you get humans who are, who are fallible, uh taking control of this kind of discovery and you have these sort of overzealous um surgeons like um uh Walter Freeman who who really just took it too far. And it was the problem there was that kind of risk benefit analysis. He was, you know, overs selling the benefits and downplaying the risks and, and actually still today, of course, we we use things not quite um you know, frontal lobe lobotomies, but there are effective treatments that involve removing portions of the brain. Uh Michael J Fox, who, who has Parkinson's, you know, one of the most famous people really who, who is, who's had the disease. He had that before deep brain stimulation was, was really uh widely used. So he, he had portions of his brain removed. I I read his um his memoir and that's a normal technique that can be helpful. Same for epilepsy. Uh One of the cases I discussed in the book as a patient who has epilepsy. And so he had portions of his temporal lobe removed actually twice. And then it did have this problem of causing him to have um some deviant sexual desires. And he ultimately started downloading child pornography and got caught by the, the FBI in the United States. And so there are these questions about, are we changing who, who people are by either removing portions of the brain um giving them stimulators that, that will, you know, continuously be stimulating parts of the brain. It can only change who they are but maybe cause them to, to even engage in, in criminal behavior. So it's just a different kind of risk benefit analysis that we're doing. And I think we always have to be very careful, uh especially when we're talking about an organ that we don't really understand fundamentally um how it works.

35:11 Ricardo Lopes: So I, I don't know what you think about this idea, but I guess that the most important thing here to consider would be. So, first of all, have the science uh right, because we need to know really what works, what doesn't in what cases and only what works should go on the market and should be allowed for doctors to perform uh in patient. Uh AND then uh informed consent because the patients really do need to know what are the risks, what are the benefits? Because there's always risks associated with any kind of intervention. And also, I mean, if the patient agrees with whatever kind of uh proper intervention that there, that there's out there, I mean, I guess we shouldn't consider that a bad thing, even though in some cases it might be very invasive or something like that.

36:14 Joshua May: Right. Right. And I think informed consent helps alleviate a lot of these concerns. Um BUT it's not perfect, you know, think about with, with Freeman and lobotomies. Um YOU know, if you, if you oversell the benefits and undersell the, the harms, then we're going to have, you know, informed consent, but it, it might not be tracking what the real risk benefit analysis is. And I think that there is, there's a danger of that always. And it's not just with, with neuroscience and neurology, but throughout all of science that we have to worry about over selling the benefits and um under appreciating the harms. And I've been quite influenced by philosopher science here, uh Jacobs who has tried to you know, raise the alarm bells here about some of this and realize that we probably the structures of medicine and partly through human fallibility and the financial incentives and big pharma and so forth. We do have certain structure that will tend towards over selling benefits and under exploring harms. He's got this great phrase, the Hollow Hunt for harms and medicine that we don't actually look for the harms as well as we should. And so we do have to get, I think the science right? And yet we're working within a structure that will tend to bias us towards um, overs selling benefits and under appreciating harms. It doesn't mean that, you know, we're, we're in a terrible state, shouldn't trust, you know, medicine and all that. But it just means that we have to be very careful, especially when we're looking at potentially really invasive and sometimes not reversible treatments that will change a person's personality and identity have profound effects on their lives. But, but as you suggest, we also have to weigh that against, you know, potentially serious benefits. A lot of these cases involve, you know, people who have severe movement disorders that can be so debilitating, they can't live independently. Uh, FOR others, it involves things like major depression that's completely resistant to other forms of treatment. And, you know, we're talking about the potential loss of life, they might actually take their own lives. Um These are potentially great benefits if we can, you know, minimize the number of people who are dying by suicide. I can have major benefits for people who can live more independently. And so I certainly understand um the motivation and for both physicians, but also entrepreneurs who are trying to develop different kinds of devices and techniques to treat these very serious neurological conditions. And I think that just it gets more difficult as we are getting to the frontiers of this research. And as we try to expand it to cases that aren't necessarily so severe debilitating or treatment resistant, I'll give you one recent example. Um THAT actually just involves an oral drug. I don't know if you heard about this, there's a drug for a LS that got pulled from the market recently called Reliv.

39:03 Ricardo Lopes: Where did that happen in the US

39:05 Joshua May: in the US? Yes, there's a company that developed it in the US and it got pulled after it wasn't really around for maybe a couple of years, but it was tricky sort of thing is again, you know, a LS is a very serious debilitating neurological disorder and there's no real effective treatments for it. And so the, the FDA, the Food and Drug administration here, the regulators are trying to figure out should we allow this, this drug even though it didn't complete um the full phase three human trials, but they said there was some early promising results and they're trying to weigh that against, but there's no real treatment for this very serious neurological condition. So they allowed it to go forward. But they said you do need to complete these trials. And recently they, they finished out the trials and they, they found evidence, you know, with much larger samples, found evidence that there was no difference between the drug and placebo. And so I think that the company did the right thing which is to pull the drug. And they said, ok, well, we did the science and it looks like it's not really working and yet it's a, it's a very expensive drug. Um, IT'S about, I think it was 100 and $58,000 a year that it would cost, um, you know, per patient if they, if they had to continuously take it every day, so very expensive drug. Uh, WE don't want, you know, people in the system to be paying for it if it's not doing anything. Uh, AND it potentially has, you know, side effects. And I think that the company did the right thing as they said, well, let's follow the science, but there are, of course, so many incentives in, in medicine to go the other direction. And just about a year ago, this, this drug, um, Sudafed pe is what they call it and, and United States, but basically AAA new form of, uh, like a Sudafed decongestant that you can't turn into meth, an illegal drug. So there's great hope that we could have a different alternative to something you can't turn into an illegal drug. And they found out after a few decades that it doesn't do anything, uh, that doesn't work better than, than, than taking nothing. But that took, you know, it took, I think it was about 1520 years. It's been on the market and it's in, it's in everything everywhere, all kinds of cold medication. So it doesn't always go that way. We don't always discover early on that something is not very effective um or that it's, it's quite harmful. So I think we need to be better about that and be especially cautious when we're talking about manipulating the human brain.

41:18 Ricardo Lopes: Yeah. And unfortunately for you guys in the US many, many times drugs, uh their price is also over inflated in comparison to other countries. But, well, I guess that's a discussion for another day. Um 00 and by the way, for the audience, I also have an interview on the show with Doctor J Jacob Stegen. So if people are interested, they can go and look it up on the channel or the website. So um moving on, uh let's talk a little bit about mental disorders because there are also different aspects to consider here. But first of all, how do you think we should approach the question? What is a mental disorder? Because this is also highly debated and even also among scientists, psychiatrists, clinical psychologists and so on So as a philosopher, what would you say about it?

42:18 Joshua May: Yeah, I think it, it still is very controversial and there's no exact consensus, I think among, as you say, philosopher, psychiatrists. But the standard rough view, I think that's pretty dominant, at least for the diagnostic manuals is something like Jerome Wakefield's harmful dysfunction view. The idea is that while mental disorder has to involve um both harm to the individual patient in some form, it's got to disrupt their, their life, their work and so forth. And also should involve some form of dysfunction in the mind or the brain. And one way to see this is, is with the, the tragic case of having homosexuality included in these diagnostic manuals for so long in the United States, it was pulled out only in the seventies and it was partly on the basis of this idea. They said, well, sure there can be harm to people who, who are gay in, say the United States where at the time uh was, you know, marginalized and it was, you know, used as a form of discrimination to, to say that people had a mental disorder here. But, but really, it's just because society is intolerant that, that we're experiencing harm. Some people who, who would say, well, yes, this is causing me problems. But um it's mostly through society and it's not a dysfunction in my mind or brain. So we always have to navigate that and, and I think it's still an evolving conversation about which things really do count as a kind of dysfunction in the mind versus just a mere difference or variation. And so I think that's roughly a, a decent conception of, of mental disorder, even if it's not entirely perfect. But these days I'm, I'm happy with, with imperfect as long as it's somewhat helpful.

44:04 Ricardo Lopes: And I guess that one of the things you were alluding to there is the issue of stigma, right? Because some of uh sometimes uh I mean, in the case of homosexuality, that's very obvious, but there are other cases where people, because they are attached a particular label. And even, I mean, nowadays, even with common conditions, sometimes people just because they have a label of uh depression or because they are bipolar or something like that, they are not treated exactly the same way as normal people are by other people. And so I, I mean, there's an issue here that the label, I mean, in terms of uh the medical system, the legal system and so on is important and it can also help the patient to know uh to have a better understanding of what's going on in their head. And perhaps that it's not their fault and so on. But also it might be very stigmatizing to have certain kinds of labels attached to oneself socially.

45:15 Joshua May: Right? Yes, I think it's a real dilemma that we have here that you're pointing to where on the one hand, we want to be able to label mental disorders, to help people get treatment, to have self understanding. That can be really important for people to say, ok, this is what I have and this is helping me understand my condition. And yet at the same time, it's also potentially stigmatizing to say that someone has a mental disorder. We don't really always do that with physical disorders. There are a lot of times people say, well, I'm sick, I have the flu. Um OR maybe, you know, I've just got a temporary physical illness or even if it's chronic, you might say, well, if someone has a physical disability and, and that can be stigmatizing sometimes, but we've come become much more comfortable. I think with realizing that we need to be aware of that potential stigma. But it's harder in the case of mental disorder because people, there have been studies on this. People feel like if somebody has a mental disorder, they are maybe um less rational uh potentially more dangerous, partly because it is a mental disorder. It's not just a physical disorder and someone who has a physical disability, like being in a wheelchair, it's not necessarily affecting their mind. And so people aren't necessarily making judgments about the likelihood of them being, you know, dangerous or untrustworthy. So I think it really is a bind that we're in. But I think there's, there's hope that we can recognize that kind of stigma and realize that we don't necessarily have to have it while at the same time still categorizing at least certain conditions as disorders. But some people will say that we just need to, to pull things out of the relevant diagnostic manuals. Um Like with, with homosexuality, the response there was not, well, we'll keep it in but try not to, to stigmatize it. Uh We say no, it's just, it, it itself is not a disorder. And so I think in a lot of cases, we're trying to navigate that um when it comes to things like like autism and A DH D and dyslexia and even deafness. So there's, there's an ongoing debate about the boundaries of, of what counts as something that's still a disorder, but we shouldn't stigmatize it versus something that just shouldn't be considered a disorder at all. And it's just part of this idea of, of neurodiversity.

47:30 Ricardo Lopes: Yeah. And related to neurodiversity. Uh This movement is also associated with ideas. Uh AGAIN, uh with the goal of destigmatize some of these things that are still considered conditions, they might not be anymore one day, I don't know. Uh uh BUT also to uh raise awareness for the fact that there perhaps are people out there who process information in different ways who think differently. We have different worldviews that are non normative and there's probably nothing particularly wrong with that. And I mean, that is the case for autism and even in more extreme cases. Like, for example, I, I mean, I, I've read a lot on these and I've, I've watched some documentaries from, for example, the Netherlands Belgium where they have medically assisted suicide. And I mean, if they don't have any underlying medical condition, even a psychiatric condition and if someone uh, declares under, uh, uh, I mean, a particular set of conditions which I can't really reproduce here because I don't, I don't have them memorized, but, uh they have to follow a particular protocol, uh if they express their wish to no longer live because, I mean, life for them is no longer meaningful or something like that. I mean, they are allowed medically assisted suicide. And I mean, I know that is a very, an extremely sensitive topic but I also think that on the other hand, perhaps is, uh, suicide should not be considered exclusively a medical issue, but also a philosophical and an existential issue. And I, and I guess that people should also be allowed to not like to live, I guess. I mean, that's just my two cents on, on that. But,

49:38 Joshua May: yeah, yeah, it's a very difficult puzzle about trying to respect people's autonomy and if they make the choice that they want to end their lives, we want to respect that. That's certainly the, the sentiment I think in the Netherlands which has the most permissive euthanasia laws, I think in the world and, and yet we're trying to balance that against uh unnecessary harm. And especially this issue about how some of the harms that can arise and the struggles that people have can arise from a society that is not accommodating or stigmatizing their condition. Uh One of the cases that came out of the Netherlands and not that long ago, could have been 10 years ago now, uh was a, a man in his forties who about my age who had struggled with alcoholism for decades and it was just, there was nothing he could do. He had been in and out of uh rehabilitation centers multiple times and he just couldn't kick the habit, couldn't get to a place where he was happy with his life. And so he, he did take advantage of um the euthanasia abilities in, in the Netherlands and and ended his life. These are very difficult cases partly because, yeah, again, we want to respect the person's decision. They're an adult um um competent to make decisions about their own life. But also you might worry that maybe there's not enough accommodation and treatment and help for, for those kinds of conditions. So yeah, bring it back to, to mental disorder. I think it bumps up against all these very difficult questions. We want to respect people's individual choices and they may construe their own condition as a mere difference and, and not a deficit. And we need to keep in mind a lot of the insights from the neurodiversity movement, which is that there is a lot of stigma out there and a and a lack of accommodations and they're trying to draw. And I think quite rightly uh some movements from disability rights activists who, who have tried to show that, you know, you can actually live a very good life with physical disability as long as there's not so much stigma and failure to accommodate. And so there's a similar, I think movement for mental disorder and, and but there's, there's questions about, you know, do we keep it as a disorder in the diagnostic manuals but just be more accommodating and not stigmatizing or do we remove it as we did with the case of homosexuality? And I think those conversations are still still ongoing for different kinds of neurological types.

51:57 Ricardo Lopes: And what about the particular case of addiction? Because that's also a highly debated topic in psychiatry and also when it comes to particular kinds of proposals that people have for particular kinds of addiction, like for example, sometimes people talk about or propose video game addiction, porn addiction and stuff like that. I mean, new kinds of addictions. Uh How do you approach that specific kind of disorder if it's a disorder at all?

52:34 Joshua May: Yeah, I think addiction is a very interesting case because it has the same sort of problem we're trying to work on stigma and make sure that we're not stigmatizing individuals with addiction, you know, for a long time, addiction was just treated as a moral failing. And there's been a move to, to really medicalize it and say that this is no, it's, it's a disorder and not just a disorder, but even a disease of the brain. And clearly the motivation there is to, to try to move away from that moralization towards treatment and, and trying to remove all the blame that we've, we've pushed on addicts for so many decades and, and for for potentially a much longer. So I think that's a noble goal. But there's a real question about whether we have enough relevant science to suggest that it is a brain disease. And this is one of my favorite topics I dug into in the book, partly because I changed my mind on it, which is a sort of miraculous for a philosopher. But I went into it thinking, you know, this, this seems like the dominant view, most clinicians, healthcare professionals, neuroscientists, even potentially most philosophers do now think that addiction is a brain disease. And the more I dug, the more I thought, I'm not sure that that any aspects of that view really hold water. So there are three main components, as far as I can tell, it's the idea that addiction is a disease, but also that it's a disease of the brain that that's where the relevant dysfunction is. But also that part of that disease is that it compromises um self control so that it involves irresistible um compulsions that ultimately lead to relapse. And there are all kinds of concerns about each of those elements. If we just take um the, the compulsion parts, there's a lot of real cases of, of people with addictions, very serious addictions who do exert a lot of control, they do deliberately abstain from using for various reasons. Um There study suggesting that people will actually take money instead of a hit of their drug of choice and, and that just doesn't make sense if it really is an irresistible impulse, you know, if you have right in front of you, the option to take this drug that you're addicted to versus just, you know, like $20 you would think that the brain disease picture would say, well, well, of course, you're just going to be compelled uh to, to, to choose the drug, but, but many people will choose the money. So there's a lot of evidence suggesting people can exert control even if it's not perfect and it often does fail. You might say, well, ok, maybe addiction is not necessarily involving always a compulsion that's irresistible, but maybe it's still a brain disease. We do have a lot of evidence suggesting that there are circuits in the brain that are impaired or, or changed at least uh after people have been taking drugs for a long time. But even that too, it's not so clear that we should think of that as due to at least ingestion of the drug. The brain disease model really is this idea that if, if you take the drug, it somehow by taking the drug, it causes changes in the brain that then lead to, you know, compulsions, uh, to use the drug. And it's just not so clear that that's really the primary source of addiction. So there are a lot of people who take drugs, addictive drug, very powerful addictive drugs and don't become addicted. Um I was kind of surprised to learn that it's only about 10 to 20% of people who even take um very serious drugs like heroin or meth. Uh THE vast majority don't get addicted. It's only about 10 to 20% or do get addicted. And it also seems to be that there's a lot of other factors distinct from ingestion of the drug that contribute to addiction, things like trauma, poverty, unemployment, uh co occurring mental disorders like depression and anxiety. It's, it's really sad actually, many people who end up using and becoming addicted, it's partly to cope with things like depression and anxiety. And so those are brain factors but they're not due to ingestion of the drug. So it's, it's not so clear that it's a brain disease. I mean, there's something going on in the brain, but it might be more a problem of trauma isolation, unemployment, depression, anxiety, these are not due to ingestion of any drugs. They're, they're sort of background factors and they seem to be really important factors when it comes to addiction. So then I started thinking, well, ok, maybe it's not necessarily a brain disease, not due to, to ingestion of the drug that causes damage to the circuits in the brain. Um, BUT maybe it's still appropriate to call it a disease. And, you know, that's also controversial. What do you, what do you count as a disease versus, you know, maybe a mere disorder. But we often think of diseases as involving a concrete physiological mechanism that we understand as a kind of uh dysfunction. And I don't think that we have that we do have this picture of addiction as largely involving the social and environmental factors and, and other kinds of co occurring mental disorders that don't necessarily have this clear underlying pathology. And so we get left with this idea that well, maybe doesn't make sense to call it the disorder. There are a lot of just, just normal brain functions at work in the background of, of things like trauma and, and poverty and isolation. So it's not so clear that we should call it a disease even if that might seem helpful for stigma. And it's not even so clear that it's helpful for stigma uh as we were talking about before calling something a disorder disease doesn't necessarily mean that we did destigmatize it. Um Actually, it sometimes exacerbates it, we think, oh no, this person has a disorder. Uh MENTAL disorders it might be more dangerous or, or less trustworthy. So it's not so clear that that's actually helping us to, to destigmatize addiction in the end.

58:16 Ricardo Lopes: And I guess that one of the main issues here when it comes to whether addiction would be a brain disease or not is uh reductionism, right? I mean, reductionism in the epistemological sense, I guess because I, I mean, there's this very common idea nowadays and it's very prevalent in psychiatry as well of trying to reduce every condition to something uh biological, something happening in the brain. It could be genes, it could be hormones, it could be neurotransmitters, brain connections, something like that. But uh I mean, it, we, I guess we should be wary of even if there's some sort of psychological predisposition and it's caused by partly by genes, partly by hormones, partly by neurotransmitters or something like that. Um We should be wary of trying to reduce all the explanatory power to those factors, right? Because there are other uh external environmental factors playing the role as well.

59:28 Joshua May: Right. Right. Yeah. And I think there's a lot of confusion about this, uh especially among non philosophers because they, they might confuse uh these different issues here and think that if you're saying there's not a reduction of the psychological to the neurobiological, you must be a dualist and you must think that these, there's some sort of sort of separate spiritual kind of world and, and these just aren't neurobiological phenomena. But yeah, that is a different debate. And really the the question here is just we could assume that, you know, the mind just is a physical thing that just is, is involves activity in the brain. But there's really a question about whether it's just brain circuits that are going to be the relevant source of explanation uh rather than more environmental factors. And of course, those are connected, I mean, think about with addiction, one of the major factors that you might think of as like environmental is stress. So people who are in stressful environments, um there's a famous study of Vietnam veterans who were in the war and they used heroin at much higher rates, but then stopped when they came home. So they became less stressed there at back home. And there too, you'd say, well that's environmental. Sure. But of course, stress manifests in the brain, there's brain activity that's associated with stress. So, so of course, it all comes anything mental comes down to something in the brain. But that's why I like this distinction between, you know, is it something in the brain caused by ingestion of the drugs or is it caused by more environmental factors? And I think that's a major source of confusion that the debate is about whether it's a physical phenomenon or something to do with the brain. Uh But which aspect of the brain and which is doing the most explanation for us should we explain it in terms of uh neurobiological dysfunction or more environmental, social things like unemployment and poverty and stressful conditions. So I think that's a really important distinction and it's one that, you know, psychiatry, as you say, a lot of them think that it's obvious. But, but it's an open question. I was just talking to a psychiatrist the other day and we were on this very issue. And so I asked him, I said, so do you think like all the disorders that are in these diagnostic manuals, do you think they will ultimately have a very straightforward neurobiological pathology like Alzheimer's or Parkinson's? And he said, yeah, sure, sure. But I thought that's kind of weird because it seems like an open question, an open scientific question. It could be that it's not uh one analogy here is with, um with race for a long time. I think people assume that of course race has some sort of biological reality. There must be something, you know, deep in our genes that differentiates the different races. But the consensus view now based on a lot of scientific research is, is that's not really right, that race is largely a social construct. And so, you know, we could have said, oh, it would be obvious, you know, that it's going to turn out a certain way. But I think similarly, it could turn out that things like addiction, uh don't really primarily involve a kind of underlying pathology in the brain and the way that Alzheimer's and Parkinson's does. But I think that there are certain cases, it might be, it could be that schizophrenia turns out that way. Uh, MAYBE not though, same thing with, with autism, maybe if we don't, we consider it just a mere difference or variation, not, not a disorder. Uh, I, I worked in an autism lab for a couple of years as part of my, my neuroscience training and uh researcher there said, you know, I've been looking for a biomarker for, for autism for decades and, and they can't find one. AND a sort of neurobiological change that they could indicate that somebody has, has autism and it's just not there. So it could turn out for any of these cases that there's just not a clear discrete underlying kind of brain difference or pathology. And I think we just have to keep an open mind about which cases will turn out one way versus the other. And that matters a lot for, for fixing and helping these. But so for addiction, which is a major medical problem right now, especially with the opioid crisis, it really matters how we treat it. So if we think of it as largely just some sort of underlying brain pathology, um that will focus our efforts on neurobiological interventions. But if we think that addiction is largely a social problem of unemployment, poverty and stress and trauma, well then our relevant interventions could be very different. Uh, WE might be looking at more like, sort of economic changes and policy changes.

1:03:48 Ricardo Lopes: Yes. And I guess that the biggest issue here is perhaps not so much that many people out there, particularly psychiatrists believe that uh everything is reducible to biology or neurobiology. But the fact that they think that that's for sure, the case, I mean, as a scientist using words, like it's obvious it's for sure like this, it is certainly this way. I mean, I guess that's no longer doing science, right? Because this might sound a bit pedantic, but uh science is ever evolving. And uh at least uh when you first get into an issue, you shouldn't go into it with particular kinds of assumptions in one way or the other. And in this particular case, assuming, assuming that for sure, all medical or all mental conditions would be reducible to biological factors, uh raises many uh alarms. Right.

1:04:57 Joshua May: Yes, I mean, I, I, I've become myself more realistic about, about science and how it works partly by doing it myself with collaborators and, and working with scientists. I think that we often have this, well, this background assumption that the science has to be completely objective and that we, we shouldn't bring in any background assumptions. I think really in the end, that's just sort of psychologically impossible that we are humans, we, you know, science is done by humans and we of course, are going to bring all of our baggage to the table, our assumptions, our biases. And I think we have to just sort of live with that and accept it, but there are better or worse ways of doing that. And I think we do need to, I think as the general public especially, uh be more realistic about how science works and realize that, you know, this does involve certain kinds of assumptions and the addiction cases. It's very evident um where a lot of this brain disease model comes from, uh the the relevant kinds of agencies that do a lot of the funding for this work. At least in the United States, the National Institute for drug abuse has been very influential in, in developing and doing research on and supporting the brain disease model of addiction. And so it's kind of no surprise that, you know, the whole institution structures are all pushing a certain framework that that's going to be the dominant framework. And I think that it sort of makes sense that that's what we're working with now. But there are, I think deep concerns about whether we're being sufficiently critical about it. Uh So I think it's great to always have an open mind when it comes to science. But I think it's also almost more important as the general public and scientists ourselves to just realize that, that we're fallible and we can't, we can't do that. I mean, a lot of philosophers of science have have come to this point. They say we should give up the value free ideal in science. It's just not possible. And, and I'm totally on board with that. I think once we look at human psychology and, and do the neuroscience realize this, this is just how the human brain works that we can't get away from our own uh biases, our own assumptions and our own motivations for wanting to seek out certain answers rather than others.

1:07:13 Ricardo Lopes: And by the way with that in mind and referring again to how certain conditions or were considered conditions in the past, like homosexuality and they are no longer considered as such nowadays. Uh We, we also have to keep in mind that science and psychiatry specifically operates within a particular socio context and also um responds to the kinds of uh moral norms that operate in the societies where they, I mean, where the med uh where the medical and the scientific knowledge is produced.

1:07:55 Joshua May: Right. Right. Yeah. And, and there are these forces that are there that are not necessarily forces for, for truth. There is, there is a lot of money involved in medicine and it's not just in, in big pharma, people often think about pills as being the relevant concern here. But it's for any any medical intervention, there's, there's going to be some kind of financial factors at play. And I think a lot of the public, you know, uh as they say, in the South. Uh uh BLESS their heart think that, uh you know, well, this is what happens in industry and, and when you have a lot of like, scientists are funded by the government that, of course, there's no biases and everything is ok, but it's not just, I think financial incentives that, that play a role in how we do science, it's all kinds of just ordinary human kinds of motivations. Uh We're still working within a social structure and we still have these kinds of, as you say, socio cultural assumptions, but it's not even just our, our moral views, like about what is deviant versus what's not. It's also just that there are a lot of motivations like, um I, I've worked in, in labs and, you know, there's a lot of motivation to say, make sure that your graduate students or your postdocs get jobs. And so that's going to influence maybe how you spin some of your research. Like we need to get this published in a good journal so that my, and that's a perfectly morally laudable motivation. Um But it might not be a motivation that's necessarily tied to the truth. Um There are conflicts there, right? Sometimes, you know, being completely truthful and, and uh honest about your, your work and its limitations will mean that you'll have costs to whether you can help the people who depend on you in your lab. And so there are all kinds of human motivations like that. Um ALSO just sort of social standing and credit, you know, scientists are human beings. And so they're influenced by what their peers think of them and they, they want to not necessarily make a lot of money. Actually, a lot of people want to do things like win Nobel prizes or get status within that profession. And that sometimes is at odds with, with the truth, sometimes the truth is not necessarily going to be as interesting. And so it's not that scientists are frauds and you know, they're trying to cut corners deliberately. But again, it comes back to full circle to, to how a lot of our choices are influenced by unconscious forces and even scientists can be unconsciously influenced by a lot of these factors to maybe inadvertently cut corners and, and not necessarily doing the best science or, or I was watching out for the limitations. So I think we have to just take that all on board, be open about it and say we've got our biases. Let's just try to read them out, let's try to understand them, but we'll probably never get rid of them entirely.

1:10:36 Ricardo Lopes: So, shifting gears a little bit. Now, I would like to ask you about a particular topic within the realm of morality that you explore in the book. Uh TELL us about moral enhancement. So what is that? And what does it have to do with neuroscience?

1:10:56 Joshua May: Hm. Yes, I mean, moral enhancement. Would just mean improving yourself morally. And the contrast there is with treatment. So there's a, a distinction between treatment versus enhancement and much of medicine is just all about treating problems, disorders, uh dysfunction disease. But then if we go beyond normal, we're in the realm of, of enhancement. And that's something that we, we want to have in morality because we are not morally perfect beings. There's always room for improvement uh individually and as a society and, and we see that, you know, with things like uh having a better understanding about, about homosexuality and about diversity, we see a lot of this improvement happening, but there's always more to be done. So I think it's, it's laudable for us to say how can we be better? And we usually do that not through neuroscience or manipulating our brains, at least not directly. But as we were saying before, of course, everything, every mental phenomenon comes down to something in the brain. So if you're going to be a morally better person, have better beliefs and better motivations and better character traits, it's going to involve changes in the brain. So when we educate Children about morality or when they read books or, or watch movies and learn something about morality, then there's a change in their brain. Um But that's kind of indirect. That's, that's traditional kinds of enhancement. We try to make sure that people are being taught good lessons and we, we learn in in various modalities just by watching other people. But also we could potentially more directly manipulate our brains to enhance ourselves. There's been a lot of interest lately about things like using brain stimulation, potentially psychedelics or, or other drugs to directly manipulate our brains in order to enhance ourselves morally. And that is a question and ethic partly because it, it might be that, you know, we shouldn't be doing this, we shouldn't be trying to directly manipulate our brains partly because it might not be safe or we might be kind of manipulating ourselves in a way that seems like we're constraining our freedom or uh maybe not being very authentic. And so we have some of those same kinds of questions about brain interventions that, but now applied to uh enhancement that maybe we shouldn't be embarking on that kind of dystopian, you know, project of trying to perfect ourselves through, you know, taking drugs and, and, you know, getting do it yourself, brain stimulators and all of that. So there's been a lot of concerns among ethicists about that. And for me, I, I'm, I'm less concerned, I think that it can be done. Well, there are concerns about enhancement. We don't want, you know, the government mandating that, you know, we lace the water with Oxytocin so that everyone becomes a little bit more compassionate or something that's certainly going to constrain freedom. And, you know, that's, that's very concerning. So I've been trying to think myself about what is a possible route of directly manipulating our brains to make ourselves better people that we could, we could still respect and say that this is, this is something that's not concerning. And I think it depends on, on the intervention so it could be their ways of doing this is very unsafe. I mean, you can, I don't want to recommend anyone to do this, but you can, you can buy various, you know, components online and, and get a little do it yourself. Brain stimulator, it's probably not gonna be very effective, it's not invasive. It's just things that you can put on the outside of, of your head and it will just give some surface level stimulation, things like um transcranial direct current stimulation, probably not going to be very effective and also may not be very harmful for that reason because it's not so invasive. But the thing I think is especially interesting is is psychedelics because those can have very profound effects on people. And yet there's a big concern about, about safety and, and psychedelics might be one of those interventions that does have a direct effect on morality. A lot of people who do have psychedelic experiences report that they have this kind of uh ego dissolution where they feel more at one with other people and the universe. And so they become less self centered, more compassionate, more open minded. These are all, you know, great traits as far as we can tell from morality perspective, so it might be that that could be a source of, of moral enhancement. And yet there are maybe some reasonable concerns about whether it's safe and whether it's something that is, is really a kind of future that we want to embark on.

1:15:20 Ricardo Lopes: Uh, I mean, whenever I hear the word enhancement, uh I get at least a little bit worried because some of that sounds like uh I mean, of course, the connection I'm making here is not immediately obvious, but it sounds like eugenics, at least to a certain extent in the sense that you're making a claim that there are morally superior and morally inferior people, or at least I would imagine that some people would make that claim and then moral enhancement would be a way of morally improving the people who are morally inferior or that would produce, for example, morally inferior, um offspring, some something like that. So, I, I mean, does that, does this make any sense or not? Is this something that probably we should worry about?

1:16:22 Joshua May: Yeah, I think that is a serious concern. We want to make sure that we are not imposing a narrow conception of what it means to be a good person. So I think to avoid that we would have to think of this as, as very individualized that we wouldn't want to say that the, the state or even the scientific community or anyone is, is saying, here's the way to be a better person and we should all enhance, you know, to this particular goal. But I do think that we could allow a more pluralistic approach where we say, well, look, everyone kind of already engages in various forms of self improvement, right? We read books, self help books. Uh We listen to podcasts. We, we study the stoics and meditation and, and those sorts of things to try to make ourselves better. And that's just indirect manipulation of our brains. We're trying to, you know, change our beliefs and values and character traits. Indirectly, I think we could do something similar with, with direct manipulation of our brains. So we just allow everyone to explore on their own, have their own sort of personal moral journey of enhancement and they might use various kinds of direct forms of stimulation to, to do that. But I hesitate to, to say it to, to as if it's a recommendation. But I do think that one possible story that this could turn into is if some people maybe using um having psychedelic experiences, at least, you know, if it's legal in their location, uh which is changing rapidly, at least across the United States, then they might use that as a way of, of exploring their own moral values and potentially enhancing themselves. And, and it's really not science. I mean, a lot of people do this already in the psychedelic community. They, they see it as not treatment for a therapy, even though that's what a lot of the research is about now, but as a way of enhancing themselves morally and to, to get to a more enlightened kind of position in their life. And so I think that is a potential source of moral enhancement, but have to be done safely. It would have to be done legally and it would have to be done with caution, with, with understanding that sometimes engaging these experiences could lead to moral regression. It could lead someone down to a path where they actually have a worse moral character traits. But it does look like it's, it's a possibility for many people that it could improve it. Uh INSOFAR as it makes them less self centered. Uh For example, so that that's one possibility, I don't know if there are others, you know, a lot of the brain stimulation I don't think is gonna have very precise and profound effects on morality specifically. Um But there could be certain kinds of experiences uh enhanced with, with certain kinds of drugs that could potentially lead people down a personal journey. Uh So that we're not saying there's one view, you know, this is the eugenic view of everyone should be moving down this path. It could be, there would be lots of disagreements. Somebody could say I'm, I'm trying to have certain experiences um maybe doing a long meditation retreat uh with some uh psilocybin as an aid that allows them to, you know, become less self centered. Uh So they can be, you know, maybe more compassionate towards other humans. Maybe another person might be doing it to explore their attitudes towards nonhuman animals to increase their compassion there. And so it might be very different views about how we're going to explore it, but it could be individual and, and pluralistic so that we don't have one single conception of what the, the right person is, which was the real, the real downfall of, of eugenics and the problem there.

1:19:54 Ricardo Lopes: Mhm. No, I really appreciate that sort of more pluralistic and individualistic approach to it because, I mean, I, I guess that even from just an ethics standpoint and even more specifically a meta ethics standpoint, another thing that could probably be an issue here, uh, is, uh, whether moral realism or moral anti realism is. Right. Correct. Because, I mean, I was just thinking that, for example, uh, what would moral enhancement really mean in this particular case? I mean, would it be, for example, someone developing psychological traits that would allow them to better follow whatever particular kinds of moral norms prevail in their specific society? Would it be a supposed set of universal moral values that people would deem to be the best ones out there? Because I, I mean, there's also this debate between the moral realists and the anti realists, right? And I would imagine that it would also, uh, play a role here. Right.

1:21:16 Joshua May: Yeah. These, these are some really thorny questions. Uh, MOSTLY I've tried to conveniently avoid but I think you're right that it's, that it seems relevant and I'll try to give you one reason for thinking. Maybe we don't have to settle those deep questions and meta ethics in order to, to pursue the project of moral enhancement. So, here's one way to think about it, uh, uh, standard kind of simple moral realist view would be something like utilitarianism. So you might think there's, there's one true morality and it comes down to which actions are going to produce the best consequences or maybe make the most people happy. Standard way of interpreting that as is a realist view. There are these, you know, objective facts of the matter and ethics and they just come down to what's gonna make most people happy. You now, even on that view, there's still going to be lots of variation and pluralism. So I, I myself, I'm not a, a devout utilitarian, but I think that many of them who, who are utilitarians would say that would say this, they'd say, well, ok, she wanna make people happy. But what makes people happy in India versus, you know, the United States or Portugal is going to be very different, just just as a matter of empirical fact, you know, there's just different social norms, there's different experiences and backgrounds. So certain things that make people happy in the United States might not make people happy in other kinds of countries or other subcultures. And so even there, you're going to have a lot of pluralism, even if you think there's still one objective fact about ethics, which is that we should make the most people happy. So I think even there, it could justify a kind of pluralism. So you won't say that we're trying to, you know, see the one way that everyone ought, ought to be, but we're just trying to figure out how to make the most people happy. And that might involve uh quite a bit of variation across cultures and, and subcultures.

1:23:05 Ricardo Lopes: So I would like to ask you now about neuro marketing. So, uh could you tell uh particularly for the audience what it is uh and whether it works or not and also then we can discuss whether it's ethical or not.

1:23:23 Joshua May: Hm. Yeah. So, so neuromarketing would just be using the tools of neuroscience to try to read the minds of consumers to try to understand what their preferences are and really how, how to, to sell them products better. And that has been done there. There are some groups like one of the largest marketing firms in the United States is Nielsen and they did have a kind of neuromarketing division for a long time, but reportedly they've closed it in the past few years, partly because it seemed like it wasn't really delivering on the investment. So there are, I think open questions about whether we have the tools to be able to really understand people's preferences by looking at their brain activity. Now, I mean that it really is a deep puzzle because you can do focus groups and all of that. But that doesn't necessarily mean that, you know, people are reliable reporting what their preferences are. You know, this again goes back to the stuff on free will and unconsciousness. So we don't always know exactly why we want the things we do. Uh My favorite example here is the ipad. I remember when the ipad came out, I like many people thought this is a ridiculous product. Why do I need something slightly bigger than the phone and slightly smaller than the laptop? And a lot of people thought this is not, you know, uh this is like the downfall of apple and yet it completely took off and I have one now. Uh AND I didn't realize that I actually would like it. There's this device that I really needed I didn't realize. So there's a real puzzle about whether you can get past what people say about their own preferences and just look directly at their brain activity to figure out, you know, what do they really want. And I don't know that we have the ability to, to get there. It's a real, as I said, a puzzle about how the neural activity in the brain gives rise to very particular kinds of thoughts and preferences, but it's rapidly advancing. So in the book, I'm, I'm very skeptical and say, you know, we're a long way from really understanding particularly the preferences. Uh But, you know, the book came out right around the time that Chat G BT got released. So that was kind of like already in press and ready to go. And then we had this major advancement in artificial intelligence and, and that's being applied now to, to brain reading and it's had some really remarkable results. So I don't know if you saw, but um so there was something recently maybe last year, a group who had a patient with a LS who can't speak anymore. And they used uh machine learning and the same kinds of like, you know, neuro network technology to decode her brain activity and convert it into speech. So she can't, you know, ver verbally articulate speech anymore. But just by thinking about saying certain sentences, they could decode it using machine learning and then produce, produce speech in a computer that's really impressive. Um Also just about a month ago, uh Elon Musk's Neuralink Company released a video of their first human who has their brain implant. And with this implant, uh they are able to uh play chess just by thinking. So this is a patient who also uh you know, can't, can't move, I think uh below his neck, I think he may have a spinal cord injury so he can't really control a lot except with, with his, his mouth. And now through having a neuralink implant, he can control a chessboard. So things are advancing very rapidly. Um Yeah, I think there are some, still some serious limitations and one thing is I like to do when it comes to neuroscience in the marketplace is compare it to what we already have out there. And what we already have out there is all these companies who have massive amounts of big data on our purchases, on our social networks, on who we're friends with. Um OUR location data with our smartphones. And all of that information is extremely valuable for decoding what people's preferences are, what they're more likely to buy or not. And, and uh for figuring out how to, how to sell things to people based just on that non neurobiological data, that's just purchasing behavior, location and social networks that is has been immensely powerful for companies. I don't know that this neuroscience technology is ever going to be cost effective enough to supersede that. I think that we could live in a world one day where as neuralink hopes, you have a lot of people who are just ordinary consumers, they don't have a spinal cord injury, they don't have a LS, they're just ordinary consumers who get a brain implant like they would, you know, wear an apple watch that uh provides feedback on their heart rate and they could use that to, you know, maybe decode some of their thoughts to get neural feedback on their emotional states. Now, that's a lot of brain data. Uh THAT of course, would just be like your location on your phone. It would just be like your purchases and your social networks and these companies would have access to all of that data. So there are deep questions about privacy if we move into that world. But I think there are also real questions about whether people will actually do that. I don't know that people are going to get brain implants uh just like they would a smartwatch, but uh it may not ever be actually as effective as all the other kinds of data that we allow companies to have already

1:28:40 Ricardo Lopes: a and when it comes to the marketing, uh bit of it, uh I mean, it also uh I think raises particular kinds of ethical questions because I mean, at least to some extent, I would imagine that the ethicists can question not just neuromarketing specifically, but marketing in general because at least to some extent it's plausible to say and accurate to say that uh it's a little bit manipulating, right? I mean, uh uh I mean, you're not really telling the entire truth when you're doing marketing to sell product services and all of that. So, I mean, it isn't it, at least to some extent a little bit deceiving and if so, isn't that also something that in the particular context of neuromarketing we should, uh, care about.

1:29:40 Joshua May: Right. Yeah, I think it's the very same general problem we have with marketing that it might be manipulative in ways that might to, to, again go back to, to free will sort of bypass our own, our own free choice. But given that I have this view that the unconscious forces in our minds are still part of who we are, still part of our free agency. Uh I think that some of those concerns are, are not as serious as they might seem at first blush, especially if we can exert control over those influences. So it's true that we have a lot of influences, especially nowadays, you know, we've got this, this attention economy where everything on our smartphone and the watches are all trying to pull our attention uh towards different features of our, our environment. And I think that does have an effect on our agency and, and our, and our freedom and we need to be alert to that and, but there are ways I think of exercising our agency in light of, of that new environment. So this is part of the, the view I'm hoping that overall the book can give is, is the complexity of our agency as involving these unconscious and conscious forces, but we can harness that in certain ways. Um I'll give you one example. So there are various kinds of reward programs that, that people have for different businesses or credit cards. If you have a certain credit card, then you'll get certain rewards. And you know, that's something that can influence you unconsciously if you have a I, that's the whole point of these rewards programs, right? So you've got an Amazon credit card. The whole point is that, you know, you'll buy more products on Amazon because you get Amazon points, right? For doing that. So that's a kind of automatic and potentially unconscious influence on your choices. But we can take some control and change the environment we're in, you could not get that credit card or you could get a different one that is for a company that you maybe would rather be spending more money at. Uh So I think you can make those kinds of choices about the environment you're in. Um YOU can do a digital detox and you know, uh delete some of your, your the worst apps that you've got. Uh THERE are choices that we can make. So I think a lot of times we can play the victim and say, well, look, there are all these companies who are manipulating us. Um But also I think we need to take some individual agency and say, but we're also kind of letting them and it's up to us to make a choice about whether we, we let them get into our, our personal lives that much in influence. You have a choice. You can, you can go into your, you know, your Google account and, and have them delete your data, you know, after a month of, of storing it or they can keep it forever. That that's a choice that we have as consumers. And I think uh what it calls for us to do is is exert more of our own agency and control over what's influencing us automatically and unconsciously and that at least can alleviate some of the moral concerns that people have about this.

1:32:27 Ricardo Lopes: So let me ask you then one final question that is also sort of related to neuromarketing and some of the other issues you raise there like free will agency and all of that. Um I mean, there's this issue of neuro hype, I mean, people hyping up certain kinds of, for example, new technologies, I don't know if you hear, for example, Elon Musk saying that through neuralink, somewhere in the near future, you will be able to have access to the internet via your brain directly or even download information into your brain, something like that. Or if you hear people claiming that if you make your kids, your three year old listen to baby Einstein, then he's like you will go up by 20 points or something like that. I mean, there, there's a lot of high surrounding products related to uh neuronal applications, neuroscience, the brain and so on. So, uh how do you think we could have a of more nuanced and balanced approach to neuro science and to the ethical side of it.

1:33:40 Joshua May: Right. Yeah, it's hard because there, there are a lot of exciting developments and as I say, we can be doubtful. But then in a couple of years, you have this, you know, major leaps in artificial intelligence systems. So it's hard to make predictions, I think for decades, people say, well, you know, we're going to have the new A I revolution and never came. And then I think in the past couple of years, people have realized, OK, maybe, maybe it's finally come where we've made a serious leap in the capabilities of these technologies. And so I, I tend to, to think we should not overhype. It's a kind of a big theme of the book is that there's a lot of excessive uh enthusiasm about some of these neurotechnology and we need to really look at their limitations and by looking at the limitations, we can also see that maybe some of the ethical concerns aren't as powerful. But I do think we need to be open minded about the, the possibility that we, we could develop technology that uh have all of these, these promised outcomes. But there are going to be built in limitations. A lot of times there's this assumption that every bit of technology just gets better and cheaper. We've seen that with, with smartphones. Uh IT'S just been radical in just my lifetime. You know how now we have what effectively would be considered a supercomputer in my pocket. Um When you just a couple of decades ago, we didn't have the internet at all. So it's, it's possible, I suppose, but there are built in limitations. So a lot of this decoding that's being done like with, with neuralink where they have these, I mean, first of all, you have to get a implant directly onto, you know, the cortex. Uh AND, and these are only electrode arrays that go into the cortex of the brain, which doesn't reach every part of the brain, right? There are a lot of structure in the brain involved with, you know, motivation and desire that would be much deeper in the brain. Uh So you have to have surgery, you know, effectively to, to even get this kind of real powerful kind of implant that can get really good signal out of all the noise of the brain activity. So that by itself, I think is, is a major limitation for, for widespread uh consumer applications. But, you know, maybe that's something people will be comfortable with, but there's also there's got to be effectively, you know, these incredibly powerful computers and algorithms that, that have to be attached to the decoding. So whether that's something that, you know, could actually be very portable that people will want to, to have on constantly is a real open question. And I think that there's a lot of fears attached to that, but we don't necessarily have to worry if it's not ever going to be a reality. And there may be people who say that they're fed up with al already, how much of our privacy has been, you know, given over to companies. And they might say this is the line. You know, I've, I've got my smart watch, I've got my, my phone that tracks my location but you could see widespread, a lot of consumers saying I'm not going to, to the brain, you know, we're, we're not, I'm not going to give that up to companies and throw it up in the cloud. And, and I think that could be a very wise decision on the part of a lot of consumers and, and it could be, could be a reality. So I think there's a real choice point for, for us as humans right now, we could draw that line in the sand and say that we want to have a certain level of privacy here. Uh And that it could actually be quite powerful and we may just reject that uh that new frontier, at least when it goes beyond treatment for serious kinds of neurological conditions. But I think it remains to be seen whether we're going to actually, you know, uh hear the hype come to reality. But in general, I think we need to, to be a bit more cautious and say, what are the realistic limitations and, and not be too overenthusiastic and correspondingly overly alarmist about the, the moral concerns here. I think there's a tendency in neuro ethics to, to think far into the potential future with a lot of science fiction scenarios and to worry about that and maybe it's just my own dispositions and personality type. But, but I'm, I'm a bit allergic to that and I like to think about the real questions we've got pressing us right now, um, about things like brain interventions, addiction and neuro diversity that we have to address now. Um RATHER than, you know, very, you know, speculative claims about a potential future. We do need to be ahead of the science and technology and think about the ethical issues. Um But I think there's a lot of focus and attention going to things that might not ever come to full fruition.

1:38:03 Ricardo Lopes: So the book is again, Neuroethics Agency in the Age of Brain Science. I'm leaving a link to it in the description of the interview when Dr May, would you like to tell people apart from the book where they can find you and your work on the internet?

1:38:20 Joshua May: Sure. I mean, maybe with all my, my concerns and qualms about uh privacy and all I might realize I'm not, I'm not that active on social media. Uh BUT I do have a website uh Josh D a.com and I have all kinds of information about the book among other things up there. And I really do hope that the, the book could be accessible to people who are outside of philosophy, but also outside of academic research. It's, it's meant to, to be as maybe overly ambitious that's trying to reach, you know, a wide audience, but also be a contribution to the academic literature. So I push my own view, it's very opinionated. Uh BUT it's also meant to, to give a introduction to this field, which I think is a really important field and it is going to continue to, to shape the future of, of ethics in the way we think about ourselves. So I've got all kinds of information up there. I've got summaries of chapters, even the case studies that I, I talk about in the book. I've got summaries of those and I continue to try to post my research on there. I'm still working on things and neuroethics and neurodiversity. Um But I'm also doing other projects um like uh on how do we make more progress on on factory farming? That's, that's one of my, my next big projects with actually a guest you had on, I think in the past Victor Kumar. So he's a collaborator of mine on this topic and seeing how he and I are thinking a lot about that. How do we understand uh the the ethical problem of factory farming and how can we make progress on it by looking at human psychology?

1:39:50 Ricardo Lopes: Great. So look, I really love the book and I hope that people from my audience really run and buy it. It's a very interesting read and also it connects to other interviews with other people I did on the show, which we mentioned throughout our conversation. So thank you so much for coming on the show. It was really fun to talk with you.

1:40:11 Joshua May: Yeah, thanks so much. I really enjoyed the conversation.

1:40:14 Ricardo Lopes: Hi guys. Thank you for watching this interview. Until the end. If you liked it, please share it. Leave a like and hit the subscription button. The show is brought to you by N Lights learning and development. Then differently check the website at N lights.com and also please consider supporting the show on Patreon or paypal. I would also like to give a huge thank you to my main patrons and paypal supporters, Perera Larson, Jerry Muller and Frederick Suno Bernard Seche O of Alex Adam, Castle Matthew Whitting B no Wolf, Tim Hall, Erica J, Connors Philip Forrest Connolly. Then the Met Robert Wine in Nai Z Mark Nevs calling in Holbrook Field, Governor Mikel Stormer Samuel Andre Francis for Agns Ferger Ken Herz J and Lain Jung Y and the K Hes Mark Smith J. Tom Hummel s friends, David Sloan Wilson Yasa dear, Roman Roach Diego, Jan Punter, Romani Charlotte Bli Nicole Barba, Adam Hunt Pavlo Stassi, Nale me, Gary G Alman, Samo, Zal Ari and Ye Polton John Barboza, Julian Price Edward Hall, Eden Broner Douglas Fry Franka Gilon Cortez Solis Scott. Zachary. Ftw Daniel Friedman, William Buckner, Paul Giorgino, Luke Loki, Georgio Theophano Chris Williams and Peter Wo David Williams Di A Costa Anton Erickson Charles Murray, Alex Shaw, Marie Martinez, Coralie Chevalier, Bangalore Fist, Larry Dey junior, Old Einon Starry Michael Bailey then Spur by Robert Grassy Zorn, Jeff mcmahon, Jake Zul Barnabas Radis Mark Kemple Thomas Dvor Luke Neeson, Chris Tory Kimberley Johnson, Benjamin Gilbert Jessica. No, Linda Brendan Nicholas Carlson, Ismael Bensley Man George Katis, Valentine Steinman, Perros, Kate Von Goler, Alexander Albert Liam Dan Biar Masoud Ali Mohammadi Perpendicular J Ner Urla. Good enough Gregory Hastings David Pins of Sean Nelson, Mike Levin and Jos Net. A special thanks to my producers is our web, Jim Frank Luca Stina, Tom Vig and Bernard N Cortes Dixon, Benedikt Muller Thomas Trumble, Catherine and Patrick Tobin, John Carl Negro, Nick Ortiz and Nick Golden. And to my executive producers Matthew Lavender, Si Adrian Bogdan Knit and Rosie. Thank you for all