The Truth About False Alarms & Emergency Warnings | Alerting Authority Podcast

What happens when people stop trusting emergency alerts? Do “false alarms” actually make communities less safe—or is the problem more complicated than we think?

In this episode of The Alerting Authority, hosts Jeannette Sutton and Eddie Bertola sit down with disaster researcher Dr. Joe Trainor (University of Delaware) to break down the science behind false alarms, trust, and human behavior in emergencies.

Drawing on decades of research—from Hurricane Katrina to global disaster response—Dr. Trainor explains how people really interpret alerts, why the “cry wolf” theory is often misunderstood, and what emergency managers can do to improve communication and save lives.

🔎 What You’ll Learn:

•  What a “false alarm” actually means (and why people define it differently) 
•  The truth about the cry wolf effect in emergency warnings 
•  How trust in authorities impacts whether people take action 
•  Why alert systems are a trade-off between over-warning and under-warning
•  How modern tools like Wireless Emergency Alerts (WEA) and AI could reshape alerting 
•  What makes an effective emergency message (and what most get wrong) 

🎙️ About Our Guest

Dr. Joe Trainor is Interim Dean and Professor at the Biden School of Public Policy & Administration (University of Delaware) and a leading expert in disaster science, risk perception, and emergency decision-making. His work has supported FEMA, DHS, the National Weather Service, and more.

🤝 Sponsored by Everbridge

This episode is sponsored by Everbridge, a global leader in critical event management.

Everbridge helps over 6,500 organizations worldwide:

•  Keep people safe 
•  Reduce operational disruption 
•  Build digital and physical resilience 

Their AI-powered platform enables organizations to anticipate, mitigate, respond to, and recover from critical events.

👉 Learn more: https://www.everbridge.com

📢 Join the Conversation

We want to hear from YOU—your challenges, success stories, and questions about alerting and emergency communication.

👍 Like, Subscribe, and Share
 💬 Drop your thoughts in the comments
 🔔 Follow for more episodes on emergency management & public safety

Transcript

SPEAKER_04 0:08

And I am Eddie Berton.

SPEAKER_02 0:10

And we welcome you to another episode of the Alert Authority. And as always, we encourage you to subscribe, follow, listen, and most importantly, participate in these podcasts. We want your questions, concerns, ideas, problems, pain points, and your success stories so we as alerting authorities can better do our jobs and make our community safer.

SPEAKER_04 0:31

And we want to thank our sponsors, and the sponsor for this episode is Everbridge. Everbridge is used by more than 6,500 global organizations to keep people safe and reduce disruption to your operations. They help leading enterprises, government organizations across the globe build digital and physical resilience by using their industry-leading solutions. Today, their high-velocity CEM platform, powered by purpose-built AI, sets a world standard for business resilience. With Everbridge, customers can confidently anticipate, mitigate, respond to, and recover from critical events. So thank you to Everbridge. So with that said, who do we have on today? And I mean they're here, but who's Joe?

SPEAKER_02 1:16

Well, only the people who are watching this can actually see our guest. But those who are listening to our podcast have no idea who is sitting. Now they know there's somebody named Joe. But let me read you Joe's bio because it's pretty spectacular. So today we have Joseph E. Trainer, who serves currently as the interim dean and professor of policy and administration in the Joseph R. Biden Junior School of Public Policy and Administration at the University of Delaware. Joe is also a core faculty member of the Disaster Research Center, where he has built an international reputation in disaster science and emergency management. He focuses on interdisciplinary research, the human behavioral dimensions of disaster and crises, and includes basic science and applied research. And some of his recent projects include disaster researcher and practitioner integration, warnings, risk perception, and protective action decision making, household insurance and mitigation decisions, willingness to work during pandemics, and multi-organizational response systems. He's led two co-edited books and has published over 45 peer-reviewed articles and numerous book chapters and research reports. Also, he has uh done public service work with in partnership with FEMA, the Department of Homeland Security, the National Weather Service, NIST, the National Institute of Standards and Technology, as well as Delaware Emergency Management and other state and local emergency management agencies. So, Joe, as you can tell, Eddie, Joe's biography, his experience, it's extensive and he brings so much to our conversation today about alerts and warnings. I am super excited. I've been wanting to talk with Joe about this one particular paper for a very long time. And I think you're gonna find out why as soon as we get into this conversation. So, Joe, welcome, friend. I'm so glad to have you here.

SPEAKER_00 3:30

Uh thanks so much for for having me having me be part of the show. Thanks, for have having me be part of your conversation. I've been I love what you guys have been doing to uh to bring science to practice. I think, you know, as a as a professor of a school of public policy administration, I care the most about that, right? I think we we want to know things, we want to learn things. But man, I I think we do our best as universities when we're when we're taking that knowledge and we're uh helping people just make the world a better place. So really, really excited that for what you've been working on and and excited to be here for for this conversation.

SPEAKER_01 4:05

Thank you.

SPEAKER_04 4:06

Well, I'm I'm really happy to to meet you. And again, for everybody who's listening. So this Joe. And one thing that Jeanette did not mention in his bio, but told me earlier, and it's confirmed by Joe before we started, he's an Eagle Scout too. That's right.

SPEAKER_03 4:22

That's right.

SPEAKER_00 4:22

Absolutely, absolutely. Scout probably the most, probably the most important uh formative organization of my childhood and and and uh big part of uh making me the person I am today.

SPEAKER_04 4:34

So definitely Well, I mean, it's it's one of those things, and at least for me, like I've from I I'm an Eagle Scout as well, and disaster researcher. I mean, that that literally in my mind could be multiple scouting events that I've been on. That's right. I've turned into disasters, but but have been great building. But if you can help me, because I am still new to the space and I'm I'm really trying to learn more and adjust my own paradigm as as we look and you know, take in the science and let science help direct our actions. So, disaster researcher that I put, I'm putting quotations here. Yeah, yeah. What what exactly is that is that and what's the distinction and why is it important to think about when we look at alerts and warnings?

SPEAKER_00 5:20

Absolutely. That's that's a great question, right? I think I think most people when they think about uh research and they think about universities, they think about what what I would say are like traditional disciplines, right? So the things that we know psychology or meteorology or engineering or that they think about those kinds of areas that are the traditional ways that we that we kind of slice and dice ourselves in universities. But but uh there most of us, I think, especially in the modern world, also work in in what I would think of as another circle that kind of cuts across in a different direction. And and and that's the fields that we work in, that's the topics we study. And and most of us spend a lot of time working with other scientists and working for with folks that are that are doing the work in agencies to in specific fields. So for me, that field is is disasters and emergencies, and and really interested in in the context of crisis, the context of emergency management, the context of disasters. And and I think a little more specifically, I think you know, Jeanette mentioned it a little bit, but I'm trained as a sociologist, but I uh and I operate in a school of public policy and administration. And what that really means is in that kind of cross-cutting world of people that study disasters, I'm really most interested in the people. So, so so I want to know and I want to understand how people make decisions, how they think about the world, how they process risks, how everyday people out there in the world decide what they're gonna do when they're faced with these tough situations. So, to me, being a disaster researcher is all about studying regular people. So that could be people in in their households, or it could be people that are emergency managers, but understanding how they how they make those decisions and and operate in that in that world. So that's to me, that's what being a disaster research is all about.

SPEAKER_02 7:16

I love that. Um, and being a disaster researcher, yeah, we absolutely cut across all kinds of fields and intersect with all kinds of different scholarship. It's it's a really fun part of being in this research space, is you get to sit in a room with super smart engineers one day and the next day super smart weather forecasters. And and then there's anthropologists who are observing all of this stuff and interpreting what's happening at the cultural level. And it's like it's just fun to see. And it's also vital that we bring together all these disciplines because disasters are so complicated. There's not just one reason that a disaster occurs. There's just all these compounding factors, and you have to bring all of that together to make sense of it.

SPEAKER_00 8:07

And there's real expertise, right? So just like just like you wouldn't, you wouldn't uh call a cop to to to uh put up fire, and you wouldn't call a firefighter to take care of a perfect crime, and you wouldn't call someone as a medic to come put their right to so people have different types of training and they have different types of expertise. And it's important, you know, if you're gonna manage a scene, if you're gonna manage the the how we understand these events, you need people that have expertise in those different disciplines to to to kind of manage the world, right? So, so so it's it's it's great to work with you and when we we do it best when we're working together, just like just like in in in any other field.

SPEAKER_01 8:45

Yeah, yeah.

SPEAKER_02 8:47

Um, so Joe, you've in your long career, you've studied some really incredibly devastating events. Um taken you to international places and across the US. Um, can you highlight some of those and then also talk about how alerts and warnings fit into the research you've done?

SPEAKER_00 9:06

Yeah, absolutely. So I have spent, especially very early in my career, I spent quite a bit of time at disaster sites. So, so that included some time in India and Sri Lanka after the tsunamis. I've spent some time in Japan a few months after the earthquake and tsunami there. I've spent time in Australia after bushfires. I spent uh some of my formative time, so my actual like when I was getting my my degrees, I was uh in in New Orleans and Mississippi and Louisiana early very soon after Hurricane Katrina. Spent time doing interviews and shelters, people to trying to understand people's experience of those events. Spent some time with the National Weather Service on service alert uh service warning teams or service uh alert teams, looking at uh the super tornado outbreaks and other kinds of things. So I spent in my early in my career, I spent a lot of time on disaster sites, talking with people who'd survived, talking with people who were responding to those events, trying to to to get us to get a sense of what was happening. And um I think that for me, that part of that was being younger, you know, it was exciting, it was interesting to be in those spaces. Uh, but uh but if I'm honest, I think after, you know, after being there for a while, there's a there's some sadness that comes from from seeing true suffering, what I think of as true suffering. And I think I think that over time, increasingly having seen the real aftermath, having seen that real suffering, I was much, much more interested in finding ways to to stop that. Finding ways to to intervene before people were having those experiences. Um and and alerts and warnings seemed like one of the natural ways to do that, right? So so if we can get people better information and they can make better decisions and they can they can uh do things to protect themselves, if we could whatever we can do to reduce that suffering before it happens, to me, that's where I wanted to be spending my time. I I think that uh I spent a lot of hours interviewing people whose lives were really, really affected. And I and I felt like I owed it to those folks to do something so that that motivated by their pain to to to help other people maybe not have to experience some of the things that they did. So that's where I got where I got into the warning space. I was really interested in what we can do to make sure people have the information they need when they need it, in the form they need it, to be able to process the risk and danger around them and make themselves safer.

SPEAKER_01 11:51

Yeah.

SPEAKER_02 11:53

Thank you. That's I'm reflecting on my own experiences of how I got into the disaster space also, and it it feels very similar to the way that you're talking about you spend so much time as as a responder in some cases. I think of being in the field as being part of that response because you're providing a listening ear to people who really need to process stuff and at some point figure out how you can serve yourself and society better by moving into the research space. And um I think a lot of us have a similar story. So I think you that's I mean, we've all three of us have experienced the direct impacts of sitting with survivors, and it's it's a hard thing. And everyone who's listening also understands that. And so to those who are listening and watching, thank you for the work that you do every day. We see you and we know that you're you're doing hard, hard work. So thank you. And that's why we do this podcast.

SPEAKER_04 12:54

Yeah, and just to kind of I literally, Jeanette, you took the words from my mouth. So I'm coming up with other words, and they are going to basically be, it really adds to the credibility um for Joe for today. And again, and again, for everybody who is listening who's not in the academic world, you're the practitioner, you're just in your office trying to figure out what's next. I I really think talking to people and listening to guests like Joe helps because, again, it's not just someone that's sitting in an ivory tower that is going, hmm, this is how it should be done, because I think it's how it should be done. Um, it really comes from experience and being out there and being on those front lines. And just as Jeanette said, like, it, it just sucks out there. Like it, it is hard. And to be able to then talk with um scientists and and people who are really trying to help us through those events and to make it easier the next time is super important. And and so my whole thing, I was literally gonna say thank you for acknowledging, you know, how hard it is out there. And Jeanette just did it even better than I did. So um Joe, thank you.

SPEAKER_00 14:08

Yeah, no problem. And I and I and it is really great to have those, to have those that you know, I I really deeply respect folks that are that are in practice. I, you know, I I uh it it means a lot to me. I think, you know, I th I think I understand, you know, though it's never been my job explicitly, but I understand those jobs and I understand what people are doing. And and I think one of the things that's great about being a researcher is, you know, I think folks, folks that are doing their work, they've got to do that work every day. They've got minutes to make life-saving decisions, there's pressure, and and I think that that they do they have it they're incredibly motivated, they're really great people, and I think but and I think you've got to make the decision and keep the keep the event moving, keep going. I think one of the one of the benefits about being somebody that's a researcher or a scientist is I can take months, years to really look at like what was going on there, what was happening, what were you thinking, what were other people thinking, what were the survivors thinking, and and and I can kind of pull those events apart in ways that folks that are doing a practice, you're just just you're just not given that time, right? I think like the the the kinds of jobs that that folks do, they just don't allow for that, right? They don't allow for that deep breakdown and debrief and pull apart. And I think that's that's that's what we do. We can provide some extra or different insights, seeing in just a slightly different way, because we have so much time to be able to think about it and break it apart and find all its pieces and parts and put it back together, hopefully in ways that's useful and and provides meaningful advice.

SPEAKER_04 15:43

Well, that's why I've really enjoyed the guests that, especially Jeanette, has invited in from the academic side, because the ones that she's selected, and and maybe it's and well, I can't say maybe, it's probably not across the board, but everyone that's come on the show, it it's it's never been adversarial. And it's never been like the guests are it's not like a compliance check or you know, report card of you didn't do this, you didn't do this. It's really been interactive and it's it's one of those common goals. Like we want the disaster, we want things to to just work out better. And so it's what can we adjust? And so I really appreciate that it's been very comfortable to have a lot of these discussions. And I mean, there I I have literally learned more doing this from being introduced, uh, and so but but that level of comfort. And so yeah, it because then like for for a question like I have here for you next is about CASA. Right? Yeah, um, because false alerts, I mean, this is a big deal for for the practitioner side. And so it's it's understanding because it you've you've studied false alerts as part of that project. Yeah. Absolutely, absolutely. And we've dealt with them. So I'd love to hear what what you've learned and what you studied.

SPEAKER_00 17:03

Yeah, sure. So, so so so that the CASA project, C ASA CASA project, was uh it was a really cool project, actually. It was um funded by the National Science Foundation, Go Science. Like I think science funding is really important for us in the in the research world. So that project brought together engineers and computer scientists, and uh it brought together folks like me and the in the that are social scientists. And the whole point of the thing was to figure out how to make tornado warnings better and try to to try to be able to predict tornadoes better, to try to be, once you had that prediction from a weather perspective, to be able to be able to have the infrastructure to communicate where that where that was happening and what the risk was, and then to understand what was happening on the on the the other side for the people who were getting that information and how they were using that to make their decisions. And and uh this project, you know, that project happened a little while ago, right? So so it's been uh it's been it's been over a decade since we uh since we were working on that. And um, one of the things that happened as part of that project is I got to know a lot more about how hard it is to predict tornadoes or detect tornadoes, how how complicated it is to understand where they're moving and where they're going. Uh the scale of tornadoes is kind of smaller than a lot of the other hazards that we think about, but the path of destruction is more total in in in many situations. And that's kind of a hard situation for for um for warnings because our ability to understand the physical threat, the hazard is what we typically say, it's it's kind of um it's not as good as the the the what people would probably want in order to make ideal decisions about how they would act and behave. So so we we talk, I talk about this as that's like a system problem, right? If if we had perfect warnings and we had perfect warning systems, and people got the information exactly that we wanted, behavior might be a little bit more aligned, but that's not what we have. So we've got to make some really hard decisions in systems like this to figure out how we're gonna communicate that information. And this conversation about false alarms was really happening happening in the National Weather Service community, in the severe alerts and warnings community. And and there was a lot of discussion about, well, maybe we shouldn't be putting so many of these out here, right? There's there's a lot of them that are going out where the events don't happen exactly the way that that they were predicted to happen. And maybe we're telling people too many times. And because we're telling them so often, we're, we're, we're, they're, they're less likely to respond. And I said, well, that's that's I mean, I understand the hypothesis, right? I mean, I read the book when I was a kid too, right? I right, I remember the wolf, I remember, right? But but being a disaster uh social scientist, I know that the process, it's actually more complicated than that, too, right? That that that there's a lot of factors that go into how people make decisions. And and while I wanted to understand how they were making sense of that, I knew that was only one piece of the equation. And and I think to to make a decision to not put information out there when you had it, that that made me just as nervous, right? Because the system, it it's it's not perfect. So you it's sort of a trade-off, right? You you're either overwarning or you're under warning. And and you've got to make a decision. Is it better to give people the information for them to process and make a decision, right? So again, that that that idea, like you as an individual, have your own decision-making criteria and your own value system. And my job as a as a as an emergency manager or as a warning person is to make sure you've got the best information I've got. Right. So that's more of a like, hey, my job is to inform you and to to facilitate your decision process. Or is the criteria, or am we really thinking about compliance, right? And what I really want is I want to tell you what I think you should do, and I want you to behave the way I want you to behave. Right. And I think then that it it's it's it's a little bit harder to just, you know, when when when the science isn't quite there, and and and you know, you uh I I tend to lean towards, well, let you you gotta give people the information and and let them process their own risks in the way that they that they they uh they can. So so anyway, the long and short of that was I wanted to really get a better understanding of of how people were thinking about this, what their preferences were, and and what it what did it mean for for the way they actually behaved. So that that's what got me interested in false alarms.

SPEAKER_02 22:00

Loving where this is going. Yeah, I as we prepared for this conversation, you know, thinking about all of the work that has been done really recently around over-alerting and the concerns that people continue to raise. And you studied this a decade ago. And this is why I'm so excited to have you talking about this particular paper that you wrote with Danielle Nagel, who's at National Weather Service, Brenda Phillips, and Brittany Scott, where it's called Tornadoes, Social Science and the False Alarm Effect. And your interest and your colleagues' interest in actually getting definitions and dimensions of these different terms, of this particular term, false alarm, is vital to our ability to think about how to do it better. And so before we get to that, I want to hear what were the definitions that people came up with about false alarm? And if you can't, people who can't see this, Eddie's got this huge grin on his face because he's like, oh, Jeanette's going into academic mode.

SPEAKER_04 23:11

No, but I want to know. Like because I know I I may use it incorrectly. Like I this is important.

SPEAKER_00 23:18

Yeah, and I think, I think, I think the the big thing about something like this is that that that it's like a lot of words, right? So we're we say words, we say things other times, but they don't always mean the same thing in the same place. And we're not used to that, right? I think a lot of us like we don't think about language in these really complicated, sophisticated ways. But but false alarm ratios, false alarm rates, false alarm. So there is a technical dimension to that, right? So the folks that are listening that are that are part of warning systems or part of weather systems or part of the that there's really there are technical ways of defining these ideas and these terms. And and I think sometimes because it's part of our expertise, we start to assume that the entire world sees these phenomena through those technical definitions. And it's just not the case, right? So that's so so it's it's not, you know, everyday, average everyday people don't think about these things as much as we do. They don't spend as much of their life worrying about them as we do, and they don't they don't process them in the exact same way. So so when when we say something like a false alarm, you know, we we might be using it in the technical side to see whether or not the warning system is doing well or not. But on the human behavioral side, what what is it we wanted to know what does it mean for an alarm to be false? What does it mean for it to be something I'm concerned about? Because again, we've got to go back to the story, right? Sometimes people call this the cry wolf effect. What we what we what we really want to know is what effect does the warning have on how I think about the warning system? And in order to do that, you've got to understand what an average everyday person thinks is good warning information or not so good warning information. And it's it's it's it's different than than what we might think when we think about indexing the system or performance measurement for the system from a technical, technical standpoint. So so when we talk to people, what we did is we just started out that simply, right? When what do you think a false alarm is? What do you how would you describe it? And uh I think uh a uh the most people, so it was about a little bit over 40% of the folks that we talked to, they said that there was a re that that a false alarm was when there was a reason to alert someone. So they gave some uh recognition of the fact that there was some kind of hazard or risk or something going on, but it didn't happen exactly the way that that the folks in the forecast system had expected it to. So, so it so so what we might think of the way we might talk about that in weather warning is like near misses, right? So, so so maybe it wasn't quite as intense, or it wasn't exactly at the same time, or it wasn't exactly in the same place, or so, but the idea was there was a a recognition that there was a there was a risk, right? There was some kind of of hazard out there, and and the warning itself was you know just off in some some way relative to that particular risk, right? So that's that's the first that was the first and the biggest group. Then there was the next biggest group, so is there 35% of folks, give or take. They they they were saying the alert or warning was made without any kind of justifiable cause, right? So some warning or alert went out, and they're really it really shouldn't have gone out, and and you know, that there was something wrong there, right? So so I think that's that group is probably that's the group that's more like what we think when we think about cry wolf, right? There was that somebody said something, there wasn't really a reason for them to do it, uh, and and they're they they said it's not justifiable. So that's about 35%. Uh there was a smaller group that uh eight or nine percent that said they actually don't know. They didn't they don't know what it means. Doesn't nothing comes to mind for them. There was uh another small group with six or seven percent that basically said that's when the it's when the the system itself malfunctions, right? So so something went wrong in the system and it sent it out. So so they but they explicitly talked about like a malfunctioning or uh or a problematic system. And then there was a small group, two, two percent, three percent, that said that I think this is more of a value statement, there's no such thing as a false alarm, right? There's like false alarms don't happen, they're not real. I really want the information anyway. So it doesn't matter, like there's no circumstance in which I wouldn't want you to tell me. So so that was a that was a smaller group. So again, you see a you see a kind of range, you see some things about the hazard, you see some things about the warning system, you see some things about their subjective assessment of whether something was justified or not, whether it was good or not. So so lots of there's lots of kind of factors in the mix here for for everyday people who are making the decision about whether or not something's justified or is a false alarm or not.

SPEAKER_02 28:07

I think it is fascinating to hear the variations in the ways that people are interpreting these things and how it aligns with the experts, the the people who are sitting in the weather forecast offices, how those definitions probably differ. Um, and I want to come back to that in just a second. But Eddie, it's time for a short sponsor break.

SPEAKER_04 28:32

It is, and I'm still going, like, oh my goodness, I love those, I love statistics. But yes, sponsorship break. And again, because we can't do it without them. And again, Everbridge is sponsoring this episode. And so when it comes to alerting and resilience, it's not just about sending the message, which you've heard on this show multiple times. It's about making sure that the right people get the right information at the right time. And from your the global perspective, you have enterprises, government agencies, all different shapes and sizes. Everbridge can help you and your organization get ahead of whatever disruption, whatever hazard exists, and protect what matters most. Their high velocity CEM platform is powered by purpose-built AI and again will help you anticipate that risk and communicate clearly with your community. So if resilience matters to your organization, which it should, Everbridge is helping set the global standard. And you can learn more at Everbridge.com, which is a link below if you are watching this. Otherwise, go to Everbridge.com and you can learn more about it.

SPEAKER_01 29:38

Yeah.

SPEAKER_02 29:39

All right, so back to our conversation about false alerts and definitions. So I think that that category, that 35% that you were talking about, that's the one that definitely stands out. I personally align with the large group of the 40 something percent that said, well, there was something there, it just didn't affect me. Like maybe it affected my neighbor across the street. But you know, as you started the conversation, tornadoes are that it's a limited, narrow path, but it is destruction in that path. So obviously, if a tornado is sighted on the ground and you get a warning, like you are at risk if you are within that polygon. So it's fascinating to me that if a person doesn't experience it, they're saying it's unjustified. Um, can you talk a little bit more about those people and those perceptions? And I think when people, when emergency managers talk and practitioners talk about crying wolf, they're talking also about people tuning out messages that are perceived as being unjustified.

SPEAKER_01 30:48

So, how do those things fit together?

SPEAKER_02 30:52

And the the dimensions, the perceptions, like false alarms, crying wolf, unjustified warnings is like it's all it it has a tangible impact on our ability to communicate directly with that particular population.

SPEAKER_00 31:12

Yeah, it's a it's a tough problem, right? And uh I think Janite, you you know this. I've I've always thought about warning systems from a systems perspective, right? And and I think that that one of the I I said this a little bit, I alluded to it. I think what one of the one of the things for folks that are warning, people who deliver warnings, I think a lot of times what people want is they want folks that are sitting at home to do the things that they're asking them to do. And honestly, it's it's I understand that. I really do. I I understand because because again, I think it's it's it's a little bit linked to that that experience. We've seen the people who have the bad experience, we've seen the folks that didn't, you know, that didn't heed the warning and ended up being hurt by it. I I think the thing that's hard for everyday people is that we're trying to to perceive that risk in the midst of your everyday life. And and there's a lot of different ways that people process that, and there's a lot of real variety in how people decide whether something's risky enough or not risky. And and I think that that in my mind, the the the job is to give people the best information we have available so that they can take it into their their minds, that they can take it into their lives, and they can process it against their way of calculating risk, their way of thinking about risk, and the decision that they think is best. And I think we need to to to recognize that that it's always going to be that way a little bit. One of the examples I always give, it's not it's not tornadoes, but one of the examples I always give to folks that are that are a little bit unnerved by people, the fact that people process in this way is like just remember the last time you were sitting in a room and a fire alarm went off, right? You didn't jump up and run to the door, right? What you did the same thing that somebody's gonna do when they when they when they get the warnings that you're sending out. You're you're saying, oh, this is a this is an important signal that something's going on here. I got to get the best information I can get possible so I can decide what I'm gonna do next. And you know, in the case of the fire, you're gonna look around, right? Do I smell smoke? Do I see people moving around? Do I hear things? Do I see fire? Do I right? That's the same kind of thing somebody's gonna do in this tornado warning, right? They're gonna say, they're gonna say, what do I see? What, you know, what are other people doing? What so again, I think, I think it's really important to get that information out there. I think it's absolutely natural for people to process that risk and to sort of take some time to make sense of it and and process it and decide what they're gonna do. So, so getting the information out to people so that they can process is really, really important. And and honestly, I think we've got to recognize that we don't have the precision that we don't have the precision in our ability to detect it. And we don't have the precision quite yet, I think we're getting we're getting there, to be able to communicate it at the level of specificity in terms of geography, so like how big or small the warning area is, to that, to to to get people to respond instantaneously every time we issue a warning, right? So so I think while that technology is evolving and developing and getting better and better every day, I think what we're what we're trying to do in the meantime is to to make sure that we give people the best information we have available. And that if there is a risk, they know that they at least have the opportunity to be to know that there's something risky in their area. Um, and then and then when we're outside of those situations, we do the best we can to use education to help people, you know, process. Yeah.

SPEAKER_01 35:01

It's a hard problem.

SPEAKER_04 35:04

Can I ask, I mean, and I we don't have this one listed, but as you were going through the statistics, and I I live in the Tornado Alley area now. I moved from California to here, and so this is a brand new phenomenon for me um over the last few years. I find myself, as you were describing that, I was in that 35% category, and then I was in the other category at different times too. Can can a person fluctuate?

unknown 35:31

Yeah.

SPEAKER_04 35:32

Am I just abnormal? Or because I'm thinking, well, false alert or false alarm, I can totally see one and I felt that way. But then I've also felt maybe some of the other ways at different times.

SPEAKER_00 35:42

Yeah, of course, right? Because it's situational. Because because because um whether or not so let's let's use different words, right? So whether or not that's whether or not that warning is relevant to you, whether or not that warning is uh uh is telling you about a risk that you think is dangerous for you, is it's really situational, right? So and so so if I'm at home and my shelter's 10 feet away from me, you know, I might think about it differently than if I'm at work and my kids are at school and I have to, right? There's the because because the things that I have to do, the the the things that I care about, that there's how what how they're spread in space and time, how the threat is spread, means that it's like I'm gonna see that same event, even if it's exactly the same. I'm gonna see it differently. Because what I would need to do to ensure that the people I care about are safe is gonna be different. It's gonna have a different time scale. It's gonna have a different, right? So, so so we're we're we're you might you might perceive it differently.

SPEAKER_04 36:48

No, that makes total sense. I have literally been in that situation where we'll go into our basement as a family. Everybody I need to, I have arms around, like we're we're good. And I'm it's much more relaxed versus I have kids at different schools, and I'm finding myself in a shelter at a primary school or an elementary school with my daughter in one of those reinforced bathrooms where everybody has to go for a tornado, knowing that my other kids are in other schools doing the same thing. Definitely. Um, no, I I think that makes total sense.

SPEAKER_00 37:20

Um and even the even the tornado itself, sometimes we're not like different tornadoes, our ability to actually detect it, to predict it, to see its movement and and rotation, to see its kind of path and x where we expect it to go. It's it's so again, so so we talk about that as like the detection system, right? And and imagine like, so let's imagine a world where we know all that, even right. So like take away the science limitations. Let's imagine a world. Now, can we actually tell people if we knew where the tornado was going, if we knew where what street it was gonna do, what was what it's our ability to actually communicate to people where that thing is, where it is right now, where is it gonna be in 10 minutes? We like imagine that game that you've played as a kid sometime where you have to like make a peanut butter sandwich and you're trying to explain to somebody how to make a peanut butter sandwich and they're not right. It's it's kind of like that same thing. If I were trying to tell you where the tornado is going, I've got to use signs and signals around you, I've got to use references you understand, I've got to, your ability to perceive where it is in space is connected to your understanding of that place in space, right? So like it's it's a really, it's a much more complicated set of decisions than than than I think most folks used to. And and I think we're getting much better at it now. I mean, the system, what when we were doing this study that we're talking about a lot today, that was you know, 10, 12, 13 years ago, that we were still, we were still at county-based warnings back then, right? So we were when we were warning for this, we were just transitioning to polygon-based warnings. So, so we were mostly telling people like something's happening in your county. And and and like even that, like that's that's a really big space, right? Right. Like that's a that's a really big area to try to to understand and personalize that threat uh for myself as an individual in my house and the people I care about.

SPEAKER_04 39:15

So and when we look at it from a local level, um, because that's ultimately, I mean, we live in the local space, um, trust is a pretty big deal. And I think that's one of the things that you also measured because we we have our local weather providers, and I mean, everybody has their different ones or the places where they get the information. And and so that's the organization, you know, whether they're the ones issuing a warning or providing you with the information. So, what have you been able to find out relating to trust and then our perception of what a false alert may be?

SPEAKER_00 39:51

Yeah, it's a great, it's a great question, Eddie. And and in this in this study, we actually found that trust matters, right? That that the extent to which a household understands who the person giving and Jeanette, I mean, this is you're you know the uh so it's you know, I'm on this this this year show together, and and I think folks know Jeanette's kind of one of the the preeminent scholars in this area in the world, right? So, so so people's ability to to trust the messages, part of that's connected to who the messenger is, right? So, so understanding the source of that of that of that warning material matters because that source that that uh affects how we think about things, right? And that's that's for for for formal authorities, that's for family and friends, that's for right. So, so so when people trust that source, when they understand that source, when it makes sense to who that person is, it's much, much easier for them to to say, look, you know, that person's doing what they're supposed to do, they're giving good information, they're right. So they they they're less likely to perceive it as a false alarm. So again, you know, we know this relationships matter, right? Your connection to community matters. The ability for people to see you as a as a as an authority for giving risk information matters because that trust, uh, they're they're willing to say, look, you know, if that person said it, if that uh uh uh agency said it, you know, it there was probably something there. And and and you know, I I I'm less likely to be skeptical of it, to to label as a false alarm.

SPEAKER_02 41:25

Yeah, it's so important thinking about how all the different pieces fit together. It's one of the questions that we get about who should be issuing these messages. And I remember Dennis Molletti's research where he always said that the person who's most trusted in the community is the person who shows up on the fire truck. And they are they are the everyday hero who puts themselves in in the place of danger to help to save you. And it is that trust relationship of recognizing that they are there in the worst days of your life. Um, and those are the people that we trust to be our spokespersons to communicate with us when we're at risk. Um, and finding those people in your community are so important. So, you know, we talked a few weeks ago with Ashley Morris, um, who's down in Texas, who is talking about the importance of creating that network of people within the community that have connections to people who are not trusting of local government organizations. And that is absolutely vital to making these warning systems effective. Um, and so it's not just the perceptions of risk and it's not just the message content or all of those technologies, but it's also the social connections with people who help to make your messages go farther. Yeah. And it's not just about going farther, like you were just saying, it's those are the people that help you to take action.

SPEAKER_01 42:57

That's right.

SPEAKER_02 42:58

And and that leads to my my question about behavior, because I know this is the thing that people are like, okay, false alarms, trust. So what do we know about people's behavior once they've experienced something that they call a false alarm?

SPEAKER_01 43:14

Like what is what does the research say?

SPEAKER_00 43:16

Yeah, interesting. Um, and again, I think of that this I'd say this is a complicated question.

unknown 43:22

Yeah.

SPEAKER_00 43:22

And I would say, I would say right now, as a scientist, we never like so I gotta I want to be real cautious. We don't we usually don't look at one study, right? So I think when as scientists, when we when we give advice, we give guidance, what we're trying to do is look across the body of science, right? So so it's not just what does my study say, it's what does my study say, and how does that compare to what other folks are have studied and what other folks have learned. And I and I would say in this area, it's a little bit of a mixed bag still. I think we've got a we've got a lot to learn here still in in terms of in terms of what the exact effect of false alarms are. So so I think what we're What we're seeing is uh perception of false alarm does matter. The connection between the more technical measures of false alarm. So so the rates that we were talking about before that agencies often measure and think about for themselves, the connection between those rates and people's perception is actually not super good. Right. So so a lot of the science that's out there that I'm aware of, the studies I know, they actually show that the that when you ask people about how frequent false alarms are, how serious they are, how and and you compare those to rates of false alarm as they're measured by agencies, there's not a lot of correlation, right? There's not a lot of, it's not easy, it's not easy to predict people's perception of false alarm based on the measures that are currently being used. So I think that's a signal. That's important. So I would start with that, right? So so what we how we measure, what we measure matters. So I think if if there isn't a connection between those things, I think it's worth us spending a little more time thinking about how we measure it then, right? So so so to make sure that our agency metrics are relevant to the people that we're actually trying to serve. So I would start there. That's there's a I think and I would I don't have an answer, unfortunately, but I but I understand I understand enough to know that it's a problem that we need to study more. And then on the other side, the question is it's kind of complicated again, right? That there the the connection between the technical rates and people's behavior, or the connection between their perceptions and people's behavior. So so again, if I think there's more false alarm, do I act more or less? Or if there, if I live in an area where there are more false alarms, do I act more or less? I would say that the results are actually kind of mixed. So so in the study that we're talking about here, we did see a small effect, meaning statistically significant. So that means that that that's a pattern that that we wouldn't, we it was an associate, it was it was from a statistics perspective, we saw something that made us say, yeah, there is something going on here, right? There is a link here, but but the effect size was pretty small, meaning that it does have an effect, but but it's not it's not changing behavior a huge amount. So it is changing behavior, but how much it's changing behavior, how much it's driving that behavior is actually a little is is pretty small. So, so so so what does that mean? It means there is an effect, right? It's not, it's not all it's not all smoke and mirrors. It's not like we should not care about this. We should care about it, but maybe it's not exactly, it's not, it's not the determinant all by itself the way that some people think it is. So I think, I think, you know, uh we we've got to put it into the broader context, and we've got to think about it as one of the factors we adjust, measure, control for. Yeah, uh, but but but I wouldn't think of it the way that some folks do as the determining factor, like this real, this real crisis that we have to, I would say that we have to be more thoughtful in how we interpret it, and we have to be more thoughtful about uh how we act, especially when we remember that we it's not like we can just be more accurate automatically, right? So the alternative here is not it's not more false alarms, or it's not less false alarms, more specific alerts. It it's the trade-off means that we're actually gonna have more missed events, right? So so it means that that we're deciding between warning people more often and warning people less often where there might be a real risk. And I think I think we've got to think about that, right? That there's a you know, I think at the end of the day, if we don't exactly know, I think the job is if there's a reasonable amount of risk, or there's a re I think the job is to let people dis like to put it out there and and and trust that they're gonna process it and make a decision. So that's a lot of talking, but that's complicated problems, complicated conversation.

SPEAKER_01 48:03

Yeah.

SPEAKER_04 48:05

I know it sounds complicated, but at the same time, I'm processing through all my experiences going, okay, how could I have done this better? Or if I'm presented with this new situation, how can I take this information and adjust my decision making to be more effective? Um, and I mean there have been changes throughout the process of alerting, um, from the time you did the initial study, which I believe you collected data like 2008 through 2010. Even though as I'm hearing everything from it, I'm like, this is relevant now, it is relevant today. Um, but that was before WIA really impacted the alert and warning space the way it has. If this research were done today, like what are your thoughts on how WIA might affect the results of the study?

SPEAKER_00 48:52

Yeah, really, it's really uh hard to tell, actually. I think, I think, I think here's what here's what I would say that there's a there's a um there's a difference in wireless warning systems, both Wii uh, the kind of more brat broadcast type systems, and I think in particular, the the what was more traditional at this time or right after this, the opt-in uh sign-up type systems that that that are that are real common. I I think when when we were when we were doing this study, it was still mostly broadcast type systems. So, so think about you know warnings going out through the TV, warnings going out through other radios, warnings, right? And then there was there were some there were some folks that were doing these opt-in systems, right, where where you know agencies had their own their own warning folks. Um, I think this opt the choice to opt in and the choice to opt out of the message of the content, I think that's that's it's a little bit different. It's not it is something that I think we've got to think about more. I think it's something we've got to to get more information about. I think, again, actually, I think probably I'm probably not the expert to talk about. I think we're sitting next to the expert to talk about to talk about Jeanette, I think has done some work in this in this space. Uh that opt-in and opt-out really changes that because because that's that's taking people in and out of the information stream permanently, which is really different than is the in what are they doing with information in the context of one specific event. So, so so the ability of people to opt in or opt out of the information stream, that's that's kind of weighty, right? That's something we need to think about. And and uh I think that's that is different than than what we are talking about here. Um in a in a different paper, I I talk a little bit about this because I think the future of warning systems is actually very much, much more individual user specific. It's much more customized to their context, to their preferences, to their to their needs. And and you know, I think we're not that far off with AI and apps and phones where you can set your preference, and I have your geolocation, and we can, right? There's there's a lot of things we could do to tailor the way risk content is delivered to folks so that it is more meaningful to them, right? So it captures the signs and signals that they care about, that it's it it takes into account their thresholds for being concerned and not concerned, and balances it against those of like experts and and people that work in in warning systems. I think that's the I think that's the challenge of the modern day, right? It's it's it's how do we structure those systems? When do we give users control versus when do we give uh system, you know, system owners control, versus when do we give individual folks giving warnings uh control, right? So I think there's there's a lot of factors in these kinds of uh kind of systems that I think are are these are the modern questions of of of warning science, I think.

SPEAKER_02 52:07

Yeah. And um, and I suspect that the concerns that existed because of the weather radio going off and broadcast messages going out, um, it increased when wireless emergency alerts started going out for tornadoes. And uh I'm honestly not sure where that conversation has ended within the National Weather Service. I know for some events they've said, no, we we can't issue those messages because they were issuing too too frequently. I think that was severe storm, and I think and blizzard, there were some blizzard messages that went out in the middle of the night that it's like, well, maybe that doesn't need to go out, maybe that's not timely and relevant to enough people that they should be woken up about snow. And that's one of the challenges with Wii's, is is while it's very, very targeted, it's if you're trying to reach people who are driving on the highway about future snow, that's not the same population as those who are sleeping in their warm homes.

SPEAKER_03 53:17

Exactly right.

SPEAKER_02 53:18

And so that's you know, one of the challenges of the channels. Um, the AI. Yeah, I mean, it is coming. And this personalized information, you know, one of the things that you and I have talked about in one of your other research projects that you've done is about whether or not people even know what they need. In some cases, we don't even start to think about a problem until somebody asks us about it. And so that's our challenge also when you ask people for like for opting in. They have to decide, well, do I want an imminent threat versus a public safety versus this versus that? It's like, what does all that even mean? So our ability to select the kind of information that we need, the thresholds, the severity, the kind of content that we would want.

SPEAKER_01 54:11

This goes back to another thing Dennis used to say. Maybe the public doesn't know what they need.

SPEAKER_02 54:20

I mean, that's a challenging thing to say because it he really prioritized the knowledge of the experts of determining what should be in a message that goes to the public. Because as experts, you understand these broader conversations and bring that to public understanding of what the public needs. Um but it is challenging because it it really pushes right up against individual senses and like you can't tell me what I need because I know it's like, well, maybe maybe you don't. Maybe you don't. So with thinking about that, um, I know you didn't measure messages, but what what are your thoughts about what should be in the message? How should in particular tornadoes, let's just think about tornadoes, how should those be communicated in today's modern society?

SPEAKER_00 55:16

Yeah, absolutely. So so I I'm gonna, I'm gonna, I I I I love what you just said. I think about the the warning system complications and pro like I think we totally it's a it's a it's a complicated problem we have today. And I and I think one of the one of the cool things that we're starting to be able to do that we couldn't do 10 years ago, 15 years ago is that a lot of a lot of our stuff was based on surveys, or it's based on, you know, interviews or focus groups or what all this kind of stuff. I think I think the cell phone in your pocket gives the ability to get a lot of other kinds of information that doesn't require your cognition to get involved. And again, there's privacy issues and there's other kinds, but but there's so like there's so many factors, so many types of information we can use to understand how people are processing risks and where they are and what they need. Anyway, there's there's so much, I just think that there's so much possibility in the future of of getting better, more specific, more tailored kind of content. And to your point, you know, though, Jeanette, like the the the which factors and who gets to control and which the I mean that it's such a complicated problem. And but it is the in my mind, it is one of the most important right now science questions. Because if we want that system five years from now, 10 years from now, we need the information and the study and the analysis of these factors now so that so that we're ready by the time the technology cat w catches up, so that we can we can build a better system. So yeah, okay. But coming back to the thing you actually asked me, sorry. Um look, we know we know that in in the content of the look like there's uh again, I it feels weird to be telled saying this. I know it's for it's for the for the benefit of folks here, but like this is this is stuff that you're so good at, right? So so so again, we we need to know who's sending the message, right? We need to know what's the agency, what's that source, right? What who is issuing this warning and why. We need to know what the hazard is. We've got to know what what what is that? Is it a tornado? Is it a hailstorm? Is it wind? Is it we got right? We've got to know where it is, right? So, so so so we've got to understand that both in kind of physical space, but also in reference space, right? So it's not just you know, there's a tornado here. It's you know, that the people need signals to help them orient to what that space actually means. They need to be able to know, you know, in a more in a more colloquial kind of way, space and time. So that's where is it relative to specific population centers, or where is it relative to, right? So location really matters. We gotta know when, right? So like the time dimension. So when is it happening? When do I need to act? What do I, right? So so that that timing part is is important. We gotta know the impacts, right? So we have to know, we have to know what what is gonna happen. What are the consequences of this type of event? What kind of thing could I could I see? And then most importantly, in my opinion, we gotta know what to do, right? We gotta know what the guidance is, right? What what specific action are you suggesting or asking that I take? So so in the case of tornadoes, that's most often shelter in place. It's most often you know, kind of move to the to the low space in in your building or move to your shelter. Or, you know, if you don't have those shelters, I think the current guidance is still to kind of go to a bathtub with a with a um with a uh mattress on top of some kind of you know some kind of protective device. So again, I think we've got to give people that full suite of information so that they that they have as much as they can to be able to make good good choices and to protect themselves and their families. And separate from this, this is the mitigation person in me, we've got to get more shelters into more buildings, closer to people, so that people have the opportunity to take protective actions and and to be safe, right? Our our building codes don't necessarily require these structures and and uh they save lives and they're not as expensive as people think they are, particularly if we're building things when we're building new new structures, and and we need to do more to make our infrastructure uh protect ready to protect us when these risks come. And and uh I think honestly, as someone who's who's been out there, I think people who don't have those things in place they really regret it. Right? They regret they didn't spend that money, they regret they didn't invest in in that in that before the event to create the conditions to allow them to be safe. So I I in addition to warnings, I'm really interested in mitigation and how people make those decisions and the way that the way that we use uh our codes and our systems to to make people safer and make our infrastructure safer. So I'm I'm gonna pitch that because it's something I'm passionate about too.

SPEAKER_01 1:00:11

So many more questions that we could we could go on.

SPEAKER_02 1:00:14

We've we've been going for a while, and um Joe, I have I've loved this conversation because it's just so incredibly relevant to everything that's happening currently. I you know, this is one of the great values of having this long history of research that's been conducted over so many years is we can go back to these things that become foundational to newer studies. You know, the work that Michelle Wood and I have done on over-alerting recently to try and get more precision around these constructs and definitions. It helps us to measure things. And that measurement becomes so important for helping to design better strategies and systems like we've been talking about today. Um, so my pitch would be we need more money for science.

SPEAKER_00 1:01:02

Absolutely. And honestly, and and honestly, forums like this, I just want to say about Jeanette and like, thanks for doing this. I think I think that uh science on the shelf is not always the best, right? So, so so so and and it's not as it's not always easy to get the things that we've learned, the things that we know out out in ways that are that are meaningful and connect with folks that that can put it into practice. So, so thanks to you for creating a a space that brings some of this information to folks. And and honestly, thanks to everybody listening, right? I think people that I always I say this a lot, time's the one thing you can't get more of. And and uh really appreciate people taking their time, investing their time in their craft and in their in their fields to learn something different, right? It's not not everybody does that, not everybody uh takes that time, not everybody is focused on that. And and the the folks that are out there in the world consuming this kind of information, doing everything that they can do all the time to get better at their craft, uh, a lot of respect to those folks for for for the dedication it takes and the energy it takes to to to to be out there in the field every day, to be constantly thinking about this, constantly evolving to make to make our world a safer place. So just a lot of respect to the folks that are listening to.

SPEAKER_04 1:02:21

Well, and then the practitioner side, just want to throw it out there. Look, for any practitioner or administrator of a program that's listening, again, what you heard today is a good reason of why we're doing it. We, as he said, take the science off the shelf. These papers have been around for a long time, yet we haven't applied them. And I'm saying that shotgun approach, right? We bit greater we, maybe some of you have, but overall we haven't. And so when I go around, when we go around and we teach, we talk to people, and we hear we've been doing this for like 15 years already. I really want to say, and they're not doing it correctly, they've just been in the work of it. And I'm like, well, what you're telling me is you've been doing it for 15 years incorrectly, and this is your chance to make adjustments. And and I'm saying that as someone who has done it incorrectly at times. Like it is, it's not a negative to say that. It's what what I think the big negative is, is when you're given opportunities to to glean information from the actual source. It's I feel like I'm I'm learning from the old Cliff Notes version or from like the cheat sheet, you know, from when we talk to the actual author or one of the authors of these papers. It's like, okay, what are you gonna do with it? Like, is now you have the opportunity. And so you can look back and be like, okay, I could have, I should have. And again, I've been there. But what are you gonna do today differently in your program in the way that you alert? And some of it may take time. I understand changing policy and procedures, but there are little things. And so I know that my program that I'm still over here where I live, every time I talk to guests, um, my program adjusts because I need to do it differently. And that's the whole idea of having a living program. Um, so I am not adverse to saying a harsh thing to practitioners out there who are refusing to make changes because that's dumb. You you need to make adjustments. Um, you're better than that. And this is just a great opportunity to do that. So be on that leading edge and again, just take that next step, whatever it may be.

SPEAKER_02 1:04:31

So there are some things that we've been doing to try and bring this kind of knowledge to communities, the ways that we've been helping as the alerting authority and as the warn room. Um, we provide trainings to communities, um, come out and spend a half day, a whole day with you, talking about your programs, about the science, about how to apply the science. Um, we conduct message audits, um, reviewing the messages that you've sent in the past to help you to see where your gaps are, to make improvements, help to review your templates that you've developed for messaging, um, policy assistance. And most recently, we ran a warning boot camp helping a group, a small group, online five days, two hours a day to review the templates that they have created and to develop new and better practices based in the research record. Um, we are going to run another boot camp in July. And if you are interested in that, please check out the Warren Room, the website, and send me a message and let me know you're interested. Um, it was an amazing week of learning and real um solid work with feedback from your peers and with experts who were on the the um the call with you. And also, as BRIC funding is now the no fo just rolled out again this week, there is an opportunity for you. As you're putting together your proposals to include investment in infrastructure, and alerts and warnings is part of that infrastructure. So truly, um, you know, this is an opportunity. Uh, if you have not worked on your alerts and warnings, it is your first line of getting to your community in an emergency situation. And it is one of the things that truly makes a difference in people's ability to take action to protect themselves. So please, you know, reach out and uh consider including that in your future um activities within your agencies. Um, Eddie or Joe, any final words as we sign off?

SPEAKER_00 1:06:42

Just thanks. Appreciate it. It was it was a great conversation. Thanks for again, thanks greatly for.

SPEAKER_04 1:06:46

Yeah. We're grateful for you, um, Joe. We're grateful for you as a guest. We're grateful for everybody out there, as he mentioned to you, for all the work that you do. Um, and if we're asking you and inviting you to make changes, it's because we really care about you and we want you and your community to be safer and better each day. Thank you. And just we know that every second has a story, and we say that a lot, but it's true. And we want to be here to help share yours.

SPEAKER_01 1:07:17

And that's it. Very good. Did you end the recording? We're still going. Hi, Adam.