#34. How Virtual Reality is Reshaping Neuroscience - Dr. Chris Maymon

On this episode, Dr. Christopher Maymon, Head of the Virtual Reality lab at Victoria University of Wellington delves into the captivating realm of VR and emotion research. Explore with us as we dissect groundbreaking studies and innovative experiments, uncovering the intricate interplay between VR experiences and human emotions. From  negative emotions like fear and disgust to the the more obscure positive emotions like awe. Dr. Maymon talks us through how VR is reshaping research and how it can be incorporated in treatments of phobias

https://people.wgtn.ac.nz/christopher.maymon

Transcript

0:00

Bye! Well then, welcome everybody to the Smooth Brain Society. As you can see, Maria is my co-host again today. Well, not everybody can see it because those listening on audio can't see it, but Maria is my co-host again today. And we're gonna, and last episode, we spoke to Gina Grimshaw about emotion and emotion research and she kind of teased our guest on today, Dr. Chris Maimon, who... does work on virtual reality. So he is a lecturer at Victoria University of Wellington. He focuses mainly on the application of virtual reality technology in psychology and has done a lot of work with emotional states and presence and runs the virtual reality lab at the university at the School of Psychology. So welcome, Chris. Thank you. Thank you, Kiotokoto, and good morning, and thanks for having me on. It's great to have you. So where do I start? Okay, first question I should ask you is, how does it feel to be the best looking person in the School of Psychology? Whoa. No way, man. No way. Now that I've left? Yeah, now that you've left, maybe. No, I mean, I don't know. I guess I'm a really, really tall person. I think that does a lot of work for me. I think I'm the most noticeable person just because six foot six guy walking down the hallway you tend to notice them. It's true. On a more serious note, could you give us a bit of a background into how you got into virtual reality research? How I got into virtual reality. Sure. So I was completing my doctoral dissertation here at Victoria University of Wellington in psychology. And I was studying under associate professor Jason Lowe. And I was focusing on something that didn't really have anything to do with virtual reality at all. I was focusing on how adults make use of their theory of mind, the theory that we all have that other people have a mind. And through a complete accident, I came upon a virtual reality system that was being set up down in our little, well, it's not little, our hub of the university, which is like our indoor quad for those who might be familiar with American universities where they have quads. And so they were setting it up for an open day where prospective students were going to come in and just hang around the university. And it was really just a toy, really. It was just kind of like, look at what we have. And I got talking to the people who were setting it up and this is 2016. And so this technology had actually been very recently brought to the commercial market. So VR has been around a while, but affordable VR has only been around really since like 2014, 2015 really. So this was pretty early. I hadn't seen a system like this before. And I asked them, I said, hey, where does this system live when you're not, you know, using it for the, I think like the two open days that we have in the year back then. And they said, yeah, outside of those two days, it sits in a closet. And I said, okay, well, if my professor is happy with me doing so, I mean, would you be willing to like store it in our lab instead? And they said, yeah, we don't, that's fine. And so what I ended up doing was just treating that. So it was the VR system and the computer that you could use to like create virtual reality on. Um, so it wasn't just like a game system. It was like a, you know, create creation studio. And, um, Jason was like, Oh, sure. Why not? We, it doesn't take up any space. Go, go for it. Put it in our lab. And if it can be something, it can be something. And, uh, I started just using it to, um, take breaks from my, my PhD and do something creative, which I was. very much longing to do. And I would just borrow students that were in the hallways around my lab and say, check this out, check this out, and put them in the VR. And I create these little scenarios that had kind of surprising twists. And what I would find is that there is sometimes very surprising responses that people would have. So I'll give you one example. I forget who it was. I invited a post-grad student who I knew into the lab, and I put them in the VR, and I created an exact replica of the room that they were standing in VR. So it was this bizarre thing where you put the headset on and you were in a cartoonish looking version of the room that you were actually in, and all of the furniture was mapped exactly. So like if you saw a desk, you could like put your hand on the desk and you'd feel it. And... what I had done for just, just for fun. I had found this little like fire simulation and I put it in the wall behind where they were facing. And so suddenly they hear like the crackle of a fire and they turn around, they see this big fire behind them on the wall and there's smoke and everything. And, um, the person who I put it in into the, um, virtual reality, they hear it. They turn around, they see it, they jump, which is what I expected. But when they took the headset off, they asked me how I made the fire hot. And I did not do anything to, you know, I didn't include heat at all. They just sort of had this idea that. That they, they thought that it felt warmer. They thought that I had done something to increase the temperature around where the fire was. And I've encountered just a bunch of little examples like that of surprising, uh, psychological phenomena that appear when you put people in virtual reality. And I've been chasing that kind of curiosity ever since. Wow. That's a fun way to stumble on to something really cool. I get, yeah. Yeah. And, um, and I, well, and so the way that it became like my, my kind of career is that I went around to all the professors in the school of psychology towards the very end of my PhD, because I realized as many people do in their PhD, I realized that, uh, I had done all the work in that particular research area that I really cared to do. Sometimes you come to the end of a three-year PhD and you go, actually, I'm done with this now. I want to do research, it's just I'm done with this area. And I was like, okay, VR is the thing that I really feel passionate about. And so I talked to Matt Crawford and I talked to Gina Grimshaw and I talked to Jason and I talked to a bunch of people. And most people had some ideas. Some people differed in terms of how feasible they thought those ideas were. And lo and behold, Gina and Matt Crawford really seemed to, this resonated with them and the areas of research they were interested in going down in the future. And especially with Gina and her interest in looking at emotional states that we could induce in the laboratory. And that's what led to the collaboration between her and I. Nice. So, then should we go straight into what that collaboration is? So it's based on Gina's work, which is an emotion. I'm assuming you're using virtual reality to kind of link up and do some emotion work. So what would that kind of look like? That is correct, yeah. So, yeah, Gina has been in the area of, cognitive and affective neuroscience for like, 30 years and is a real world leader in that field. But I was very new to emotion research. I hadn't really delved into that area of research. So I was in this odd position where I was coming into her lab in like a just finishing my PhD role, but I had less knowledge about that area than. some of the most junior members of her lab. And so I had to kind of catch up. And I mean, I didn't know everything about virtual reality. I knew enough to create some things and to set up the system. But it was still a rapidly advancing technology. So the point is that at the beginning of this collaboration, I felt like only slightly more qualified to do this than anybody else around me. And so there was a lot of learning to do. Um, but yeah, what we ended up, uh, doing with this, uh, tool is, um, trying out some commercially available games that were, um, becoming popular because of these really, um, remarkable responses, emotional responses that people would have to them. So. You may have seen videos yourself that have come across like, like Tik TOK or other social media of, you know, people putting their friends in VR and they're in a simulation that's called Richie's plank experience. And this is a, this is a video game that came out in like 2017. And it's simple. You go into an elevator in VR. This elevator goes up. Um, it opens at like the top floor of a very high rise building. And there's this wooden plank, like a plank off of a pirate ship, if you imagine. And, um, You walk along this plank and if you like step off the plank, you'll experience yourself falling all the way down, um, to the ground floor at a fairly, um, realistic speed and nobody, nobody forgets that they've got a headset on of course, but it's, um, if you look up some of these, these videos of, um, of some people experiencing this, their responses are, are huge and, um, and, and they are. more authentic relative to other ways that we might induce emotions in the laboratory without virtual reality. Just because they're really powerful emotional responses and you can see it in their behavior and if you attach like a heart rate monitor you can see it in their physiological changes and if you ask them how afraid they are you can see it in the magnitude of their like subjective emotional state changing. You know, if you compare that to the standard way that we might induce or aim to induce an emotion in the laboratory before virtual reality, what people would often do is use emotionally charged images. So you'd look at something that might have the picture of a, something that would be a threat if it were really there in front of you. When you do that, you know, you're part of your brain is going to appraise it. It's going to recognize it as a, as the threat that it is and not just a picture of a threat and that part of your, you know, your brain, your mind is going to respond to it as if it is really there. And, and that, you know, in the more, you know, top down consciousness, kind of parts of your brain are going to sort of realize, oh, that's not real, but like there's going to be like a response that you can't stop almost, so it's going to have emotional. value of some sort and then you can kind of see differences. But the problem with that methodology is that you are assuming that if you find an effect when you have this what I'm gonna call a weak emotion induction paradigm, where it does significantly change things, it may change them in the direction that you predict, but it's a pretty small effect, that The results that you have in those studies are going to generalize to the situations where we are in an emotional state in real life. So when you're experiencing fear, because there's a person behind you when you're driving on the freeway and they're clearly experiencing road rage directed towards you, that is a moment where you experience fear in a way that is not like the fear of seeing a picture of a spider on a screen. And the magnitude of the response that you're going to have is different. And so the, my kind of rationale for using virtual reality to advance, um, affective neuroscience, um, these, let this laboratory research is that. It allows us to move closer to a science that is more naturalistic and that matches where the testing conditions. match the real life circumstances that they are meant to be representing in the lab in some way. But because you in virtual reality have complete control of the context, you could like change anything programmatically, you can retain the same internal validity, the same experimental control. that you would want in these traditional cognitive psychology experiments where people are in front of a computer looking at images, right? Like you have really good control of the parameters of that task. Well, you actually have the same control of the parameters of your virtual reality. And that's really quite, I think, I think that's quite a big breakthrough because for the entire history of psychology, it's seemed... like of psychology research as we, as we know it, it's seemed that there is an, uh, intractable, uh, disconnect between those two goals. You can either have an experiment that's well controlled where you can set all the parameters exactly, but by virtue of doing that, you're going to remove the circumstance from being anything like the real life circumstance you're going to, you're going to, uh, apply the results to, you know, that you're going to use it to like predict what would happen in a real life situation. And if you wanted a real life situation in the past, well, you usually had to sacrifice experimental control. So these will become observational studies and you could never get to like causal claims or evidence that could support causal claims because you simply wouldn't have the ability to control enough variables to have a good experiment. And so it's like for decades and decades, it seems like that is just the state of things. And now with VR, and I don't think there's any other technology that's allowed this to happen, you don't have to choose. You really can have both at the same time. So how different are the... Oh, sorry, not how different. How similar are the responses to real world in that regard from your experience of working with VR? Yeah, that's a great question. And there is some variability depending on the emotion that we're talking about targeting, but... If I want to give you a broad strokes kind of answer, they look pretty similar. So on the plank, they're not, so I should say that we've been using that plank walking simulation that I described to you earlier. We don't use the same video game, Richie's Plank experience anymore. We built our own in Unity so that we can control more of the parameters, but... It's the same general idea. You go in a plank. I mean, you walk a plank that is up, up high. Um, and, and you see the, the lethal drops that would happen if you step off the plank, off the plank. Um, in that situation, my participants and I've run about 700 or so people through that simulation over the last five years or so, and people's responses. Uh, are I, they're very like what I might expect in a sincere height experience. Uh, the reason that I'm being like cautious with my words here is because I suspected if I put them in exactly that situation in real life, almost nobody would even step onto the plank. Um, in fact, I don't, I don't know if anyone would because there's no safety equipment that you can see and there's no reason to assume that the plank is is not going to just fall away. So I want to hesitate to say, oh, it's just like real life. But it's way closer to real life than the other things that we might do. And I do see a good number of participants. who just cannot step onto the plank or they'll step onto the beginning of the plank when they, the elevator, you know, the virtual elevator is like very close behind them, but getting out to the end of that plank is not possible for them. Or if it is, it takes a while. So the planks like three meters long, two and a half, three meters long. So it doesn't take long to walk there. And, um, some people take about like a minute and a half to walk that and they'll do it in like. 30 or 40 steps, which are tiny little shuffling steps. I've seen people who will get down and crawl on their hands and knees. And then a lot of people just stop halfway and go, please end the simulation, I really can't go any further. And so we always have to kind of, at the end of each study, we have to go, okay, how many people in the data set that we have reached the end of the plank? And it's always like about 10% of our participants who didn't quite. make it all the way. Yeah. So it's not exactly like what I would expect to happen if I could put them into the real world situation, but it's pretty good. And it's probably not that different from the fear response that we all experience in situations where we actually don't know if we're in danger. Thank you. Thank you for answering my questions while talking. No, I don't have quite, I don't really have a question to ask other than, aside from these scenarios, what was the most outrageous thing that you made people believe? Kind of like you said, that they forgot that they were wearing the headset. Yeah. Actually, there's a study we're about to run. And it's led by uh, one of our master's students in the cognitive and affective neuroscience lab, um, which is the lab that Gina runs. And, uh, this student's name is Nat and she is, um, interested in using virtual reality to induce, uh, not fear, but disgust. And I think this is a great emotion to use virtual reality for, because, um, it allows you to. bring participants really close to things that would be like dangerous to their health possibly if you brought them that close to it, you know, or like difficult to like have those, those things on hand in a lab. Like you'd have to like be prepared to like bring out like something like horrendously moldy and you know, so it's a good, you know, it's one of those good situations for VR where it solves lots of practical problems. But Um, what we've done in this study and, um, the data collection is about to begin on this one. So I don't know. I won't have like results for you, unfortunately, that I can talk about, but I can tell you how we're going to do it. Um, because I'm quite, I'm quite proud of the, yeah, the it's gone through a lot of iterations. Um, and so what they do is they go into a regular room. with some books on the shelves, just like a small waiting room, if you will, in VR. That's the first thing they say. And there's a door. And then at a certain point, that door opens. And so like we always have a neutral environment that precedes our emotional environment. Same in the plank, you're standing on the city street before you go into the elevator and go up to the top. And so in this one, you're in this waiting room. And then it opens up to a bathroom. And this is like the most abused bathroom you have ever seen. Like this is like... never been cleaned before and definitely used before and found by all the bugs in the world. And we actually work with a really, really talented VR cinematographer in Wellington. He lives in Patoni. His name is Ed Davis. And he is the guy who's responsible for creating a lot of the original digital assets that we've incorporated. So he literally like let food rot in his, like one of the rooms of his house. And so that he could like photograph it from hundreds of different angles, because you can use a process called photogrammetry where it will take images, it being the computer and create a three-dimensional object that treats those images. kind of like puzzle pieces, like a three-dimensional puzzle, and it will assemble it and go, oh, with all that data, I can like show you and texture the object that you are photographing. And so he's done this for like different bowls of food that he's let become really moldy. And so he's brought those assets in and that is in the bathroom, like on the ground or like in bowls in the sink, you know, and there's these like... bugs that he has animated to be flying all over the place and you hear them. Um, and the, and then there's of course, there's a toilet that we have put in the room that is just full to the brim with some horrible substance. Um, and then what we make participants do is walk towards the toilet. Um, and when they do so, we, we have some aspects of this scene. that are meant to make the simulation more than just an audio and visual experience, because a big part of making people feel any emotion, disgust or fear or anything else, is that you have a multi-sensory integrated experience that like, you know, you can feel things, you can see things, you can touch things, you know, you can sometimes smell things. you know, whatever seems to be appropriate for the circumstance. You want to, you know, represent all of those aspects. And so what they do, they, they're asked to take their shoes off and then they walk across this, it's like a spongy mat, but we make it wet and, um, and, and so it's squelches under your feet and you know, you feel that liquid on your, on your bare feet and in the VR, what it looks like is you're walking across this like carpet. that's saturated in some horrendous, you know, oozing liquid. And then, so you get past that, right. And then you are asked to kneel down and take your hand in VR and put it into this toilet, you know, put it right into the liquid. And what we've done is we have a bucket that has warm water in it and we've put like silly putty in it and you fish around. And so, you know, in VR. They just like, they're putting their hand into where they see the, this thing. But what they feel is that like water or liquid comes up to meet them. They have to like fish around through stuff. And the idea is that, I mean, you can tell, you can probably tell what the idea is. You know, I mean, it's just becomes very convincing. And the idea is that we can make them feel as disgusted as I think any other study could possibly and ethically do. But of course, we're not actually putting them in harm's way. And we're not actually making them do something that is actually as revolting as it looks like it is. And the best thing is we can tell them that it's a disgust study ahead of time. We can say, hey, we're gonna try and disgust you. Doesn't matter, usually. Like fear, disgust is a very phylogenically ancient response. We've had it probably... for many, many millions of years where if you encounter something that you recognize as a contaminant, there is a disgust response to keep you from ingesting it. And so, yeah, that's probably my most, my favorite, the study where I'm like, we are using this technology exactly the way I think it should be used. How much do you have to pay them to do it? You know, well, that's an empirical question that I'm going to have to, that we'll find out because we're going to see it. Can I answer that? Well, yeah, sure. Can I answer that? So, so Natalie is a friend of mine and I did that for free as a test subject. I had to go through the experiment for free a while ago. Well, people are really interested in their own emotions, I find. Well, they're interested in VR as well, but like everyone's interested in how they're going to respond, uh, you know, unless you're in a situation where people know that they really, you know, hate something like they really hate, you know, spiders or something, but like lots of people are really curious just about what could happen even if they, they know it's going to be gross, right? Yeah, true. I think. I think the VR lab, and this could also be an advertisement for your lab because this thing goes out to places. So as a student, I've taken part in a few of the VR labs activities, their experiments, and it's really fun. I don't know if the plank one was as fun, but it's very interesting to know and quite a few of the other students also whenever we see sort of volunteer work, what do you say? some trial work for the lab going on. Usually most students put their hands up to try to find out what it's like. It's generally a fun experience, uh, or at least a memorable experience. Uh, so yeah, we, we don't have too much trouble finding participants who are interested in being in our studies. What we do have trouble with is once people take part in a study, um, well, like if it's a, if it's a plank study, for example, where they're walking the plank, they can really only be a real participant in that study once because, uh, I suspect that your past experience being on the plank changes your subsequent experience quite dramatically. Now I think that people need to actually do some empirical work to see, to, to support that, to substantiate that feeling that I have. Um, the only real reason I think that's the case is that I've been in the plank simulation, like hundreds of times myself and I developed it. And when I'm developing it, like sometimes I'm in there, like, you know, uh, hovering over, you know, this great X, this great drop and I'm on my hands and knees, like seeing if the real plank aligns with the virtual plank and I don't feel any fear at all and I'm, I am afraid of heights and I, you know, it's just gone for me, it could never make me feel any fear anymore. And so, you know, experience does play a role, but we don't really know how fast that drop-off is. So, and that way it becomes hard to find more participants. Would you argue that if you had a fear of something and kind of like exposure therapy in the sense that if you did it multiple times in a real world situation as well, after a point, you'd kind of get used to it? Yeah. Assuming your fear isn't that high that you can't get beyond the first couple of times. Um, I'm glad you asked, I'm glad you asked this question. Um, because that's probably that that's one of the most successful applications of VR, um, ever in my opinion is that there's this, uh, people use VR for exposure therapy and there is actually quite a, quite a good bit of research showing that outcomes are just as good. for those who have completed their exposure therapy in virtual reality, then those who complete their exposure therapy without virtual reality, which is extraordinary because if you have a phobia of like spiders, an exposure therapy might have to like, like an in-person non-VR exposure therapy means you have to have like a spider on hand, you know, at such. Maybe not at first, maybe at first, like the first trial is like, Oh, here's something that looks like a spider. Here's a image of a spider depends how. Power or how strong and, or how, I guess, well, how developed the phobia is, I guess, but, uh, you know, to, to complete those, um, those, um, exposure therapies, you end up having to get to something that is, you know, really is the object of your fear or like for people who have, who have fears of like, going on an airplane, like you have to bring them onto an airplane. People who are, have, have a fear of driving over bridges. You have to like eventually get them to drive over a bridge. Um, and there are all sorts of limits on, uh, just the practical availability of these things, but a VR headset can just be anywhere you need it to be, and you can have VR exposure therapies, uh, paradigms that are phobia you can imagine really. And they're all like the same price, you know? I mean, it's just like a, it's just a software onto any headset, it'll work. And so the fact that it seems like it's just as effective to use VR, I think is a massive step forward in, it doesn't make exposure therapy itself a better therapy, but what it does, Is it just, it broadens the scope of for whom can this therapy can be for who can get this therapy? And so it's one of the, it's one of the like biggest, I think success stories of, of VR. You talked a lot about invoking, well, sort of negative emotions in your VR studies. Do you have any interesting studies that invoke positive emotions? Yeah, we do. So I mentioned earlier that I think that disgust and fear are like good emotions to induce in virtual reality. Some aren't. Some study, I mean, some emotions just aren't really suitable for virtual reality? Like, or, or I don't want to, the thing is I don't want to like say anything that could, uh, that's just a, an artifact of me not imagining it. Cause you know, it's a, it's a creative medium. So I, I'm going to say this, but I'm sure someone will prove me wrong in some years, but like emotions, like guilt can be a little harder to induce in virtual reality because first you have to like convince somebody. that whatever made you guilty in VR is something that actually matters. That's not just like a simul- you know what I mean? So you have to like, but like if you're guilty because you've caused harm to somebody in VR, you have to believe that there was like a person in VR that you've done something to, you've wronged in some way. And so these just become more complicated to do. Whereas things like fear and disgust, like just seeing the depth cues that tell your brain, that's a drop-off, that's a cliff. That's enough. It doesn't have to be really realistic. It doesn't have to be really sophisticated. And just seeing things that say, I know that to be a moldy bowl of food, that's enough. Your body's, your brain's like, hey, I say no more. We don't, I don't need to question anything. I don't need to question reality any further. Um, so there are some emotions that are more appropriate to look at in VR than others, but when, so, so one of the emotions that is a positive emotion that I think is a really appropriate one to look at in VR and in fact we do is the emotion AWE, A-W-E. And so this is, um, part of our ongoing, um, Marsden work that Gina and I have been doing since 20... 20 and it's being led by her PhD student, uh, Ren Pritchard, who, um, has been working with us since she was an honors student. And what we've done is created, I'll just, I'll describe the VR scenario that we use. So you go into the headset and you find yourself in like a, a chrome metallic interior of a sort of dome shaped. object and it's like a spaceship and there are these levers in the spaceship and you can, you can like walk around it and you can use your hand controller to press a button that operates one of these levers. And when you do the, they're like these kind of like, you know, you know, suicide doors on a, on a car that kind of open like a wing that's like the, the windows all kind of open out and suddenly you see that. you're surrounded by windows, you know, above, below, and on all sides. And, um, you see out to these different expanses. And in the earlier days of, of this research, uh, we didn't know which kind of environments might be best for triggering awe. So we made a couple of them with our, um, colleague, uh, that I mentioned before, Ed Davis, the VR cinematographer. And, um, In one scene, you'd be kind of looking out over the top of this really large mountain range, you know, all around you. As if, like, as if you had just put a spaceship on top of one of the mountains in, like, the Milford Sound, and you have that, like, endless kind of expanse of mountain tops. In another scene, you open out, and it's like you're in some distant galaxy with these, big, colorful fictitious planets and comets that fly through the sky. And, you know, there's the sun right above you. That's really close. And, uh, and, and yeah, you can just kind of perceive of this grand vastness and, and the other, and the third, um, and final scene that we tried out or that we created, um, is as if you've basically just brought your spaceship to, uh, a shallow reef. teeming with life underwater. And so, you know, there's that expansive blue that you get when you're underwater. And then there are these whales and, you know, orca and different fish all around. And we pair this experience with like, what we describe as awe-inspiring music, like kind of like symphonic, you know, stuff you might have in the background of like a trailer for some, you know, heartwarming movie or something. And, um, and, you know, this is what people say is, is part of the awe experience is that you just need to perceive some vastness and that could be literal distance, or it can be like conceptual vastness. Like if you, if we could explain strings theory to somebody in a way that everyone could appreciate, um, presumably they would experience awe the conceptual vastness of understanding it would have the same effect. But of course that's really hard to do. Right. Um, and the other components, there's the vastness component. The other component is called a need for accommodation, which, which means that something about what you're experiencing is, uh, causing you to update your understanding of the world. Like it's challenging your beliefs about what is possible, you know? So that's, that's like, you know, when you're in space, for example, like You shouldn't be able to be here. And, you know, if you're underwater, like, you know, how did I get here? All these things that are like, how is any of this happening? That's also part and parcel of the awe experience. And we've, we've run a number of people through these simulations and, um, we're pretty confident that it's, that it's doing what we want it to do, but the challenge with an emotion like awe is that, and this kind of connects to your previous episode with Gina, where you talked about emotional responses, not all emotions have equally identifiable profiles of responses. In theory, they all create a response subjectively, physiologically, behaviorally. But some are easier to capture. than others. So fear is pretty easy to recognize because heart rate goes up. Your skin conductance, like how sweaty you are, that's going to increase. You're going to say that you're more afraid, obviously, but also you can see it in body movements. People become more rigid, um, in their movement, they've become more cautious. They move more like slowly. Um, you can recognize people as being afraid usually, even without complex signal processing and statistics. With awe, it's much more complicated. Um, it, it doesn't really map evenly onto, you know, a strong negative or a strong positive emotion. Um, in fact, some people think that there's, that they're separate emotions. There's positive awe and there's negative awe. Um, and there's, there's debate about that, of course. Um, but it doesn't present as clearly in physiological responses. And so we really The only clear way that we've been able to show that this environment induces awe is just by asking them how much awe on a scale of one to 10 they're experiencing. And we do see that, you know, that they'll say that, oh, very high awe on average in these scenes, relative to the neutral scene, the neutral scene being the spaceship when the doors are all closed. But, you know, I think we're, we're much more in the early days of that. branch of our research. Um, uh, and that's one where I think VR is going to really help that research going forward to try and put people in these environments that, okay, we know that people think they're odd. Can anyone find out how that's changing the state of your physiological condition? Because that would be super helpful. You know, I've got quite a few questions to follow up on that. The first is why did you pick awe over other positive emotions? I know you said that It's probably a good one to look at and so on but I guess because you could pick happiness something Joy, whatever are those concepts just hard to measure in general because for something to be happy, Generate or you find something funny. Usually it's a social response kind of like guilt where you need to have that connection make it funny or are there other reasons why you went with awe? Especially because you said it's harder to ascertain what awe is in terms of physiological response. While you could tell if someone's happy by looking at them and how they act and so on. That's a great question. The reason we chose awe, there were multiple different reasons really. There's, on the one hand, there had been some research that was already being done, um, where people were using virtual reality to induce awe and yeah, it just seemed like a really appropriate application of the technology. And that was the basis of our, of our project that we were, you know, collaborating on was we want to induce these emotional states. We want to use virtual reality to do this. We should choose emotions. We feel most confident will be. Um, induced well in virtual reality. And part of that calculus was who else is doing research in these different emotion areas. And so we found people such as Alice Chirico is one of the researchers who does a lot of work in awe. And she uses virtual reality. And so we based a lot of our. So like when we came up with those three scenarios that we decided to create in the lab, they were based off of papers that she had published in 2017 and 2018 that showed that, yeah, people feel odd in these, you know, mountainous environments and space environments. So we said, okay, we have some precedents for that. Um, also there's a lot of, there's, there's been a pretty, uh, interesting expansion of work into awe, it's become a more, a much more popular and interesting emotion to study. And for decades, there was really no research on it. There was some theories about it, but it hadn't really found its way into laboratory science. And the, there's a lot of interesting things about it. But one of the most interesting things is that when people experience awe, it's suggested that you're more open to new perspectives, that you're more likely to change your attitudes or your beliefs. And this is one of the aspects of it that I find really intriguing, because I'm just looking around the world and popular media, and there's all this talk about, you know, the division between different groups of people and all we live in such a, you know, a polarized world. And we need to find, you know, compromises or we need to see each other's perspectives more, or some people's attitudes and beliefs just need to be updated. Like, you know, sometimes there's no right answer and that there's just two different opinions that are equally valid. Sometimes. That's not the case. Sometimes you have situations where we need to educate people because we can't keep having huge swaths of the population failing to acknowledge this fundamental reality of our world. And so the idea of what is a mechanism through which we could possibly encourage people to not just to come around to our views, but to like... change their beliefs and their minds at all, in whatever direction. And so I'm thinking, I'm not the only one, there's a lot of people out there who are doing research that looks at this, is maybe if we could experimentally make people feel awe, we could foster some change in an attitude or an intention or a behavior that... would allow us to kind of understand, like build a working model of how different emotions change our beliefs or allow our beliefs to become more malleable, if you will. And so that's kind of the branch of the awe research that I am particularly excited about. There's plenty of other questions about just the emotion itself. How do we recognize it? What does it do in the body? When do we feel it? Why do we feel it? Evolutionarily, why does this exist? You know, what is it for? Why is it something that we all have experienced? And when do we experience it? Those are all great questions as well. So yeah, to get back to what your question was about why choose this, there were these sincerely intriguing brand new questions and we thought, oh, this is really... This is the, something is starting. And I want to be part of it, you know, now happiness and joy and it's all very. Important to understand, you know, if we could make everyone feel a bit more happier, if we understood how people become happier, that would obviously have tremendous benefits. I think when it comes to happiness though, it's a bit more of like a, a vague emotion. It's a, you know, lots of things can make you happy. And if you describe yourself as happy, you know, some people describe themselves as happy when they feel. relaxed and content. Other people describe themselves as happy when they're busy, when they're a little bit stressed. Other people describe themselves as happy when they are going to bed. You know, it's like, there are all these different things that make people feel happy that finding a single scenario that you could present in the lab that would make the, you know, anybody who comes in feel happy. That could be a tall order, actually. Yeah, no, that, that makes sense. That makes a lot of sense. Uh, so just to pick up on a few words that you said, which I found funny. So you're going to use virtual reality to change people's actual realities in terms of all. Yes. I mean, I think that's, that's a, if there was a headline that came out of some of our research that said that I wouldn't be too upset about it actually, like as far as headlines go, that's not a bad one. Um, because that's the whole idea is that it's a virtual reality and yes, they know that they're in a virtual reality, but your brain doesn't have. You know, we were talking, I, I mentioned evolution before evolution takes place over a long period of time. And most people don't have, you know, any experience in VR or a few experiences in VR, no, there's, there's very few other instances in our phylogenetic past where we've really had to question. whether all of the sensory experiences that we're having are a accurate reflection of reality. Usually you just trust that they are. And so when you're in the headset, what your brain is doing to construct a representation of your reality in all of its many forms is not different to how it's constructing a representation of reality when your headset's not on. So yeah, I think calling it a reality and saying, like, you know, shaping your reality and virtual reality, whatever, this is not an inaccurate statement, inaccurate statement. Yeah. Is there... So I guess the funnier way to ask this question is, how do you know we're not in a virtual reality right now? As we were saying, is there any real difference between the way our mind perceives the two things as far as we're serious guys? Well, I mean, we are. I mean, I think we are. I think it just depends on what you mean. So, for example, do I think that like... for, you know, you've seen the matrix, I'm sure. Do I think that we're plugged into some computer, uh, that is actually being like run by machines or some entities and that like, we're not like our actual bodies are not here? No, I don't think that. But what I do think, um, is that this feeling that we all have all the time, I have it right now, you have it right now that you are embedded in reality. that you are a direct perceiver. Like think about the quote, like the eyes or the window to the soul or whatever. Like your eyes are not a window. This is not a hole that you're seeing through. It's receiving electrical signals or it's receiving information that it turns into electrical signals and sends that to the brain. And all you are is the brain. All you are is your brain. You're in a dark room. You're in a box, your skull. You get incoming signals, you make predictions about the next signals you're gonna see, and somehow there's the byproduct of that prediction and that receiving of these electrochemical signals. The byproduct of that is this really vivid multi-sensory experience that is our subjective experience of reality. But... I am not like, I perceive myself at the helm of my reality. I perceive like I'm right here, but I'm not. I'm like, I'm in here and I'm only in here, you know? And so in that way, we're in a virtual reality because we're not directly perceiving all of reality. We are constructing reality. And in that way, all I'm doing is adding another virtual reality on top of the virtual reality. that we're all experiencing all the time. So what's, what's that saying? I think therefore I am kind of thing. Uh, yeah. Yeah. No, a bit like that. Thoughts, Maria? Yeah. It's, um, I mean, I mean, sure. Yeah. I can, it's, it's more like, um, I mean, I mean, I guess the question is like, is that what people mean when they say we're living in a simulation, you know, I suspect that most people think like. When, when they say like, Oh, we're living in a simulation. They mean like, there's some like potentially nefarious entity watching over us and setting all the parameters. And, you know, like we're in somebody's game of Sims. Uh, I don't think that's the case. I, I think, I mean, I don't know, like the law of parsimony says you should, you know, if you don't have an empirically testable hypothesis that can disentangle two explanations. Your best bet is to go with the simplest one. For me, the simplest one is to assume. No, no one is, no one is operating our universe. Like a, like a kid playing the Sims, you know. Well, assuming that this reality is the real one, do you have any studies where you're trying to do either augmented reality or some sort of hybrid between the real reality and the virtual reality? Yeah. Yes, actually. And it's a very early stages study into awe. Uh, cause we've, there's, there's multiple studies we're doing into awe and, um, in this one. And, uh, and, and actually it's like, it's really in like the prototype simulation phase, but the, we want to have people, um, experiencing the off scenario inside virtual reality. And we also want to, um, stress them out. Um, but we didn't want to do that in virtual reality. Um, it's going to be hard to talk about this because it's like so many things that are like moving parts and that might change over the co I'm, I'm going to present to you like little glimpses. So I don't know if this doesn't sound like the most coherent description of a study, that's, that's because like that coherence is sort of where that's what we're approaching right now. Uh, but. One of the things we wanted to control for in the different scenarios that people are going to experience in this study is we want them to be in virtual reality the entire time. And so the way that we do that, or the way that we're considering doing that, is using what's called the pass through feature on most of the recent headsets that have come out. So most recent headsets have cameras in the front or cameras in the corners. And those cameras can be used to pass. Like it's called pass through you, you can see through them in the headset. And so you get this bizarre experience of, you know, you have a headset on that's blocking your vision, but you're seeing a live video feed of those cameras in a way that makes it seem like you're just like looking right through the headset and looking out. And so that's, that's where, you know, that's the same thing as augmented reality where you might be wearing, um, those headsets that look more like glasses. where you see the real world and then digital objects are included in your view of the real world and that's what augments the reality. That's why it's called that. So we want to do this in a way where, okay, we want them to be in VR all the time, but instead of having to make a brand new VR experience that can stress them out, what we could do is just use a stress paradigm that exists. face to face that works really well. Um, but we want them to be in VR. Well, let's use the pass through feature. Now they're in VR. They're seeing, you know, the room they're in, they're seeing a person that is actually there. We don't have to like animate the person. We don't have to go through the work of creating a virtual entity, uh, an avatar, if you will, or a social agent that is so realistic that it allows people to believe that is a real person perceiving me. We can kind of, uh, get a, like circumvent that challenge by just being a real person in the room that you're seeing through the pastor, but you still have the headset on. Um, so, so in that way we're using augmented reality, but before this study, um, I, I usually kind of skip past augmented reality because, um, in most cases, one of the fundamental uh, benefits of using virtual reality is that you have complete freedom and control over how the virtual context looks. And now if you use augmented reality, you're back to people perceiving the lab room as the lab room. Like you're back to them. You have to make what you can of the physical space they can see. And then you also have to get these, these virtual objects to like, seem like it's part of the environment. Um, of course this different, this, this differs depending on what it is that you're actually interested in researching, you know, some people may, that might not be a, the most important aspect of what they're creating, but, but in general, we use virtual reality and not augmented reality because it affords us greater freedom from the laboratory. But you know, yeah, it's, it's funny you asked this because, you know, this is a very, very recent discussion where it's like, Oh, actually for the first time in our, in our lab's history, we actually are using augmented reality. Um, I wanted to kind of combine your... The thing about augmented reality is, of course, it's the vision of what you see is a lot better, as you mentioned when you were talking in the initial stages, when you spoke about the fire and so on, that it was just this cartoon looking scenery. And I know virtual reality has improved since, but how good is it now and does that? does like the quality of the actual graphics make a big difference, especially to awe, because in your fear example and in your disgust example, in both of them, you're kind of supplementing it with standing on a plank in the fear thing and in the disgust thing, they were walking on sponge and they kind of had this additional layer while if you're with the awe study, you just are kind of relying on the graphics itself of the mountain, of the water and so on, while- doing something like that in an augmented reality might be better? Yeah, that's a great point. So, I mean, I should say that if you really wanna experience awe, the best way that we could do it is not to involve either augmented reality or virtual reality. It would be to take them to some place that actually is awe-inspiring. So, like, you know, again, if I had the resources to do so, if I had just the money to buy everyone like a... that I wanted in my study to have their own personal like fly over of Milford sound. I do that. I choose that over virtual reality because I suspect that the awe experience would be much more powerful and I wouldn't have to convince anybody that, Oh, what we've done in the lab. It's probably like what happens in real life. Like I could, I could, I could point to my evidence or my results and say, you know, that is real life. I used real life. So You know, that, so you're right when you say like, Oh, well, wouldn't it be better, you know, to have a more multi-sensory experience for awe, you're right. Like I, I think it would be a stronger simulation if I had made a simulation of, for example, uh, instead of being in a spaceship, they're actually on top of a mountain and they can feel like grass beneath their feet and maybe they feel like wind that you might expect at the top of a mountain, um, that would be. probably a more effective means of inducing awe. Now there's a question about whether the environment would be what's inducing awe or the technology, like the mediated experience that I'm providing them would be inducing awe. For me, it doesn't usually matter which is the case because I just want to know that I can experimentally cause people to feel awe. So whether it's the content is not part of the question for me, but yeah, multi-sensory integration, to get to the other part of your question about like photo realism, that's the more important component than really high fidelity graphics. This is one of the things that has surprised me so much over the years that we've been using the same plank environment since 2020. And even back in 2020, it was, it was an, it was a nice environment. We didn't actually make it from scratch. It was like, uh, sort of a free demo environment that was provided on this online store for anyone to use. And so we said, oh, there we go. There's some buildings there's like, uh, as a city, we got it. Um, and, but you have, it might look, it probably feels like to people more real than they expected, but it's not going to like look. like a real world in the sense that you can very easily find things, no matter where you look that go, oh yeah, that's not exactly how that would look. You know, edges being all right angles. The real world is crumbly. There's no litter on the streets anywhere. There's no people walking around. The shadows are very rigid. They're not. There's no kind of, there's none of that like noise that, not like actual sound noise, but just sort of like statistical noise, like things not being perfectly aligned. You know, that's life. Life has all sorts of like little jittery parts of it. And, you know, so you can look around and go, oh, that mountain, I can tell that that's a shape that has like a texture on it. It's not actually. It's not green because I'm seeing the tops of trees. It's green because it is a texture laid on top of the shape. Um, you can see that. And yet what we have found over and over again is that it doesn't seem to be the case that is constraining people's emotional responses. So on the plank, people are. People, people can, some will tell us sometimes like that doesn't look very realistic and I'm looking at their heart rate and I'm going, Your heart rate has increased by like 15 beats per minute right now. Um, relative to before I put you in the headset, um, or like your hands are sweating, your legs are shaking, you know, and, and they'll, and I say, how afraid are you, and they'll say, I'm really afraid. And I was like, but you, you know, I don't say this to real participants, but sometimes we're not piloting participants. I'm just running them through to see how it goes. I'll ask other questions. I'll be like, well, you know, you know that you're in a room. Uh, that's just a, you know, a carpeted room, you know, there's no actual drop there and you can see all these ways in which it's not a real city and they, they can appreciate that and it doesn't really matter. Um, so the photo realism, I'm sure there is a threshold where if it's not realistic enough, to a certain extent, your brain is not gonna appraise it appropriately that you're not going to treat it as a real world. And the same with like the frame rate drops. So, you know, when you're seeing a VR, you're not actually seeing like, you're not actually seeing, again, not a window, you know, into something that's happening. You're getting a two dimensional image that's being sent to each eye. And that image is being replaced every one 90th. That's the benchmark that we try to hit, one 90th of a second, so 90 Hertz. And when it happens that fast, you don't perceive that. updating and if it's especially when it when you turn your body when you turn your head the image that you're getting is being replaced fast enough and updating fast enough with your body movements that it just gives you the impression that i'm looking through like into a world that is the those the parameters that matter way more than photorealism so yeah the quality of a It's interesting, but it's not the parameter that I pay attention to much in my research. But to answer, to go back to the very beginning of your question about how much better are VR headsets, I mean, they are rapidly improving and we actually just bought the Quest 3 headset for our lab and we've been playing around with it the last couple of days and it's pretty astonishing you know, these headsets are, and this is a headset that doesn't even like, you can connect it to a PC, but it doesn't need to be. It's got its own built-in operating system inside of it and it's lightweight. And, and, you know, so like, and, and what Apple has been doing with their vision pro, I mean, I'm, I'm sure this will have to go through some like iterations before it becomes like the, the version of the headset that in the future we come to see as like the real powerhouse headset. That's kind of how other products with Apple have been. The first ones, you know, maybe really exciting and new, but it just doesn't work quite as well as like later iterations. So, yeah, I mean, there's a lot of improvement. There's been a lot of improvement on the augmented reality front. So, you know, those pass-through features have been really well integrated into system. so that you've got a menu in front of you, but you also can see behind the menu, like your environment, because no one likes to be walking around not knowing if they're about to like trip over their dog or walk into a wall. And other features like hand tracking have gotten really good. So, hand controllers are fine, but you have to learn how to use them. And if you can see your hands and everyone's hands look roughly similar, I mean. five digits, they usually are in this kind of an area. And so, we all use them in relatively similar ways. We pinch things, or we twist things, or we pull things, or we point to things. And so, there's labs that Facebook, or Meta, has thrown a lot of money at to make their headset where they've just gotten really, really sophisticated on things like hand tracking. So it's those kinds of features, the ones that allow you to interact with the virtual world, keep the virtual world being presented to you at a rate that is lifelike. So like increasing the field of view so that it matches the field of view that we have because I can see my hand and then right around here, I can't see my hand anymore. So the headset should similarly have that kind of like predictable field of view. Those kinds of parameters are the ones that are gonna be critical for improving the quality of the experience. Nice. Yeah, that kind of makes sense. I look forward to buying the Apple Vision Pro 15 in a few years time. Yeah, yeah. In decades maybe. When you buy a new headset, do you sell the old one? No, no, we keep the old one. So we don't buy them very frequently. Like we had this headset from 2020 that we still use actually. We were using this Quest 3. Uh, in an upcoming study, but we're still using our old five pro I. Headset from 2020, uh, even though it's now considered a very like outdated headset, but it, but it does everything we need it to do. Um, and we know how to use it. And so, you know, there's, there's like the question of, Hey, do I, do I want to like upgrade my system, even though I'll have to like learn how to use a brand new system when the system works well enough. That's part of like, part of knowing how to navigate like a rapidly evolving consumer market of technology, because this is just going to keep happening. There's going to be like new headsets at the same rate that there are like new cell phones that come out, right? Like, it's just a, it's an arms race of VR headsets. And so you, when you choose to buy one, you have to go, okay, I'm going to buy this one, because I think that when the next one comes out, that's going to be a little bit better or even a lot better. I'm not going to resist that temptation because this one I've got is good enough. Um, so we make use of it. Yeah. We keep making use of them. Yeah, no. Nice. So Maria's effort at getting a free headset did not work out there. Um, if they were mine to give away, I had to try. Considering we've been recording for over an hour now, and it is pretty late over here where we are, shall we, did you have any final questions Maria? Or Chris, did you have some key things that you haven't touched on? Well, there's like one, I kind of wanted to talk a little bit about presence, but I think I have dabbled in that in some of my answers. So maybe if, if we have that conversation later in your like podcasts about papers that are coming out, maybe I'll save all the presence stuff for, for that conversation. Yeah, that sounds really good. Yeah. So I, so yeah, no, I'm, I'm happy with what we've talked about. Yeah. That's it. I think I'm good. Awesome. Yeah, so as Chris mentioned, we're going to try to do another set of episodes where we focus on individual papers. And Chris is publishing one soon on Present. So we will get him back on to talk about that. And we'll just focus on that research paper as is. But Chris, as you know, the final section is asking a few rapid fire questions. Do my best. Maria, I've sent them to you too, so you can ask them as well. So we'll just alternate questions each. Does that sound good? OK. Sounds good. Cool. So Maria, you can ask the first question whenever you're ready. Okay, number one, summer or winter? Oh, summer, especially living in New Zealand. I love road tripping, so yeah, summer for sure. Morning. Uh, morning actually. Um, I love mornings and I, I love getting up at like five 30 when I can make myself do so. And having that kind of feeling like. You're the, it's a private. It's like your own private part of the day that most other people aren't, aren't awake yet for. Uh, okay. Cats or dogs? Oh, that's hard. Um, I grew up with, I grew up with dogs and my, my father was allergic to cats. So I never really experienced having a cat or living with a cat, but I adopted a cat with my wife about a year ago. Or two years ago, actually almost. And I don't have a dog right now. And I am in love with this little creature. And I, so now I really, I think at this time in my life, I can't believe I'm going to say this. Cats. Oh, one of the few people who switched. It's so strange to say that. I love dogs, but right now it's my cat. Right. If you're a life or a movie or TV series, what genre would it be? Probably sci-fi, honestly, just because of what I do for work, I guess. Fair. What's your favorite food or cuisine? Breakfast. I'll just, eggs and bacon. I'm a simple man when it comes to cuisine. Sahil, how are we doing there? I think that's a pretty unique answer. Yeah, no one's ever given that before, so that's good. Something different. What's your least favorite type of music? least favorite type of music. I, I like have a respect for heavy metal, but I have never been able to actually want to listen to it. I keep every couple of years I try and you know, I, Matt Crawford is really into, um, metal and he's playing it all the time. And so when I, I'm like, Oh, I gotta get into this. And then I try and I'm just like, it's just not for me. It's just too much noise. If you could live anywhere in the world, where would that be? Aotearoa, New Zealand. Yeah, I'm pretty content here. I'm from America. So, you know, I came out here like purposefully to live in this country. And it's been the right move for me the whole time. Brilliant. Yeah, so this is a good advert for, what do you say? What's it? Tourism New Zealand, this. Totally. Alright, so what's, considering we've spoken about virtual reality, what's something dumb or stupid people have tricked you into believing? Um, there was a time when VR was really in its infancy, like 2017, where I thought that, um, like everyone would be in a headset, like all the time, like, you know, that, that they were gonna, that was gonna replace, uh, video game consoles and that it was gonna be replace the work laptop or the work PC. And that like, it was just gonna be the only technological medium you would need. And I think that was a bit of a silly thing to believe in because people like when they can like look away from something, it's a bit too confronting to be in a VR all the time. And... It might happen in the future. I don't know. That's what people are trying to do is like, how do we get people to spend more time in VR? But I don't know if it's ever going to get to that point actually. And I don't know if it should. Well, what's the thing? Elon Musk is trying to put chips in humans at the moment. So wouldn't we could just jump right past VR and straight to having microchips in our head. We shall see what the experience is for these brave, courageous people who have signed up for this. But where that comes from, yeah, no comment on that. Fair enough. Hey. Oh, okay. If you were not doing this, as in making people walk on planks, what would you be doing? I would probably take up an apprenticeship in surfboard shaping. It's an old art of hand shaping surfboards and crafting them by hand that I would, if I had the time, that's what I would be doing. That sounds really cool. Unfortunately, my only real experience of seeing some someone or something ever do it is have you seen the kids movies surfs up? No, I haven't. Um, it's, it's about a thing, the poster for it. It's about a penguin who just wants to surf and, and is good at surfing. It's such a random movie, but in that he meets this, Oh, this wise old penguin was like, if you want to learn how to surf, you have to carve your own board. And yeah, it's, uh, it's, it takes a surprisingly long time to learn how to, how to do that. And then you end up having to make like a dozen boards before you have one that. Will be properly shaped so that you can actually like transfer the motion of the wave properly. Um, it's like a really, it's a, it's a really cool, I don't want to say it's a dying art because people do it, but, um, it takes a while to get into. Awesome. Cool. Then very last question for you then, which we ask everyone, if you had any advice for our listeners for us, what would it be? Well, I guess, who would, hmm. Any advice for your listeners? don't spend too much money on a VR headset. The ones that are affordable these days are really good. Like if you wanna get a VR headset, like don't break the bank on it. Fair enough, good. That's great advice to end on, very topical. But thanks so much. Thank you, it's been a lovely time. Yeah, thank you. Thanks Maria as well. Thanks everyone for listening. And yeah, until next time. Take care, bye. Until next time.