Improve Police Behavior via Artificial Intelligence

Spending hundreds of thousands of dollars on an unproven tool is wasteful, and Artificial Intelligence equipment for policing can be just such an expensive outlay.  While there is a growing body of research showing the AI benefits and limits within policing, the jury is still out on its overall utility.  Professor Ian Adams joins the podcast to discuss his paper titled, “Improving police behavior through artificial intelligence: Pre-registered experimental results in two large US agencies,” published in Criminology (coauthored by Kyle McLean and Geoff Alpert).

Main Topics

• Using AI to audit body camera transcripts, there was an 80% increase in professional behavior in an already professional department.
• Substandard behavior was significantly reduced in a department with existing problems.  
• The improved behavior may not be a change in the hearts of officers, but better is still better.

Don't forget to like, FOLLOW, and share.  Sharing this podcast or an episode is one of the best complements I can receive, which will help grow the show.

And don't forget to provide a review.  Giving five stars is never a bad idea.

Feel free to email me your comments using the "send us a text" option (above), or at the following email address: policeinservicetrainingpodcast@gmail.com 

You can also contact me at: Bluesky: @policeinservice.bsky.social

Transcript

SPEAKER_01 0:13

Welcome to the Police and Service Training Podcast. This podcast is dedicated to providing research evidence to street level police officers and command staff aligns. The program is intended to help the police and law enforcement community create better programs, understand challenging policies, and dispel the myths of police officer behavior. I'm your host, Scott Phillips. There are several justifications for the use of body worn cameras and policing, including an assumption that use of forced incidents will be reduced, the video images will improve case processing, and the videos can keep police officers accountable for their actions. While body worn cameras may be helpful in a piecemeal fashion, the problem is the sheer volume of video images, making it almost impossible to use all of them in bulk. In other words, if a citizen complains that I use offensive language, the video is likely to prove my guilt or innocence. And this would be a single event that lasts around 10 minutes, and reviewing the footage would be pretty simple. But if a police agency wanted to audit the overall behavior of their officers, it would take months, if not years, to review the video images in order to come to any conclusion. And don't forget the overtime or the need to hire additional personnel to do the work. The use of artificial intelligence may be a solution to this problem. Now I want to emphasize may. From a personal perspective, my academic career has taught me to be skeptical of just about anything that does not have substantial empirical support. Yes, that's a personal problem, but at least I'm aware of it. But while the body of research evidence is small with respect to AI, it is growing, particularly in policing. It's as if the policing field is experiencing its own Moore's Law, in which the research evidence seems as if as if it's doubling every few years. This is an exaggeration, but it does make the point. Joining the podcast today is Dr. Ian Adams, who worked with his colleagues to examine the utility of AI to review or audit the content of body camera images to determine if the use of cameras impacted police officers' behavior. Ian is a newly minted associate professor in the Department of Criminology and Criminal Justice at the University of South Carolina. In a prior life, Ian was a police officer in Utah, and this is his second time on the podcast. Thanks for coming on today, Ian.

SPEAKER_00 2:31

I'm always happy to be here. Thanks, Scott.

SPEAKER_01 2:33

You were on the podcast about a year and a half ago and you discussed some general notions about AI and report writing. Now, why AI and body camera images?

SPEAKER_00 2:42

Yeah, interestingly enough, I I see these two as intimately connected. They're two technologies that are both using some form of artificial intelligence where the feedstock is body worn camera uh transcripts. In other words, the audio that's coming from the body camera. So in report writing, it's agencies using that transcript, that body worn camera transcript, to try to generate a first draft uh police report that they can then edit, modify, and submit. In the case of what we're talking about today, same beginning, right? We're going to take that transcript and use natural language modeling and AI to extract information from the transcript that tells us something about what's going on in the officer's behavior and even the who they're interacting with, their behavior. So ultimately, I see it as the same process ending in different points.

SPEAKER_01 3:45

I kind of hinted at this in the intro, but why is this area of inquiry of importance or relevance to the police?

SPEAKER_00 3:52

Well, uh, it's being adopted incredibly rapidly, and we have almost no empirical evidence for it. So I I heard you in the introduction talk about being a professional cynic. That's kind of how I see the job of an academic. Um, we should be I'm an I'm a techno-optimist with cynical leanings as far as it's really hard to affect human behavior in the world. Humans are tough subjects, and we should just at the beginning about anytime we have a hypothesis, we should probably assume it will fail. Um, because especially if your hypothesis is like we're going to change something about the human world, the human world is incredibly resilient and doesn't like to be changed and is on its own path and has its own uh causes, multi-causes often. So us coming in and shifting something around shouldn't really be expected to have that effect. That said, and and that was the case, by the way, in report writing, right? But I have now three or four studies in that area. Um they're all sort of, and there's other studies outside of policing that kind of agree that we're probably not going to get big efficiencies out of using AI to replace human writing. In effect, in short, I would say we're we're exchanging drafting time for verification time. That's what ends up happening. On the other hand, in this in this study, uh, which was uh just published earlier this year, uh we did find big behavioral shifts. Uh, and and I think it's one of the first studies to show that AI in the workplace, it's kind of stepping outside of policing, but just AI in the workplace uh can actually extend the reach of the of the organization. It can do something new as opposed to report writing. That's doing something that exists. But evaluating, you know, in our case, 200,000 body camera videos, yeah, um, is something that agencies can't do on their own, haven't been able to do. So that's a new that's an extension of human ability. And there we saw that we were able to um both raise the floor and and the ceiling on officer professionalism.

SPEAKER_01 6:00

Yeah, I'm thinking as you just said something, that uh one of the problems uh I've been I've been living in the world of evidence-based policing for the past few years. And uh getting officers to go to hot spots and interact with the public, do something for 15 minutes, you know, using a Copra curve, get out of their car, assuming it's the weather's nice, those kinds of things. But w one of the biggest problems that I've been noticing in the readings is that it's it's very difficult to get at what the officers are doing when they're actually at those spots. If nobody's around around to talk to, when the then there's nobody around to talk to. But you want to make sure the officers aren't acting like a potted plant either for 15 minutes. So suddenly I'm thinking to myself, if the officers engage their body camera uh and and there was it's recorded for that 15 minutes, then that might be an avenue to gather data to find out what kind of interactions are going on, what people are talking about. And again, it could be anything from where I am talking about the Buffalo Bills or the Buffalo Sabres prospects for the year. But that the point here is that now suddenly I'm thinking that maybe this is an avenue to have data to get at that behavior when it's almost impossible to do otherwise. Okay, thank you. Thank you for making me think about it.

SPEAKER_00 7:12

I I agree. When I first came into this world, you know, my dissertation was based on a prediction that this technology, the review of body worn cameras, would exist and be functional much as it is. That I mean, I'm not patting myself on the back too much. I think other people were having similar ideas at the same time. In fact, the company that the vendor that um is in this study is a company called Trulio. And Trulio's founder and tech lead, um, they they were having the same ideas at about the same time I was. They went and actually did it. I did the the scholarly coward thing and just like thought about it, right? So all kudos to the technologists out there who are um pushing forward with these ideas years and years and years in advance of their use. But they need to be depth, those effects need to be demonstrated. Right now, my main concern in this space is that chiefs are suffering a form of vendor fatigue. They are being hit every day, dozens of times, by new vendors with promises upon promises, with zero empirical evidence that they could they can achieve even their most basic promise. Um, that is the case for AI report writing, um, in which now we do have empirical evidence that they don't save time, and yet that doesn't stop certain vendors from going back to those chiefs and saying that they do. Right. Um however, in this case, in in in and you know, I'm happy one one thing I'm happy about the study is um, as in some of my other technology studies, I am not somebody at who's who's aiming to show things don't work. I'm aiming to test the hypothesis that they don't, right? That's that is the null hypothesis. So um when when in this particular study, we actually showed in terms of like scientific effect sizes, very, very large effects, right? We're showing we're talking about decreasing unprofessional behavior by up to two-thirds over a six-month trial, and we're talking about increasing highly professional behavior by upwards of 80% or so. And importantly, this is a passively driven system. This that what in effect, what's happening in our trial is um, of course, we have a control group who doesn't get any of this information, but we have two other treatment groups. We have one group we call self-mediate, meaning the information comes in, gets produced by the AI, gets sent to a dashboard that is accessible only to that officer. It's not it's not internal affairs, it's not their lieutenant, it's not their sergeant looking at it. And and I really thought it was important that I had this treatment arm in the in the study because I believe that policing is a profession. Sort of at the root of being a profession or a professional is the urge to do your craft better and to self-learn. And so my theory of change in that treatment arm is not one of discipline or correction, it's one of exposing how that to that officer, how they talk, how they interact with the public, because none of us really know, right? And in fact, one officer that I talked to um had had a comment uh when we visited the agency. This is in Richland County Sheriff's Office here locally in South Carolina, and he said, you know, I never really realized I dropped the F bomb that much. And like I was like, Yeah, me either, right? Like I'm sure as a cop, I probably dropped the F bomb more than I was consciously sort of aware of. Right. Yeah. And so that caused him, he was, he was, he was kind of a religious guy, I think. Um uh given my conversation with him. He was like, I'm I'm I'm I'm aiming to be a better, a better cop by uh that that alerted me that I needed to wash my mouth a little bit. Um so that was one treatment arm. And we saw the biggest effects in that arm, right? That that's where we saw the most improvement. And it it was good to see. It was it was it was some evidence that when officers are given information, it doesn't need to come in the form of an award and it doesn't need to come in the form of a disciplinary note in order to shift their behavior. They're professionals who want to be better at their job in aggregate.

SPEAKER_01 11:29

Right. Let's let's dig into the into your research and and because that's one of the things I noticed, the two two approaches to getting the information to the officer on their own review versus somebody else telling them. But first, let's talk just a little bit about clarifying body worn cameras and policing. Because most people, most listeners are gonna be aware of this, but are they in use countrywide, at least in the United States? Because I was under the impression that it was like 50%. But again, I only have so much time to keep up with the literature. It's it's higher than that.

SPEAKER_00 12:00

Oh, yeah. By the you know, we don't have incredible estimates right now. And I think BJS will release a technology survey a little bit later in the year that will help us um maybe pin down the latest numbers. But by 2019 or so, I think around 80% of the agencies um were were implementing at some level. Okay. Um, which is a pretty fast, you know, they didn't really start getting adopted until 2015. I I wore one as a testing unit maybe in 2012. Um, but they they didn't gain critical mass really until after um the the events in in Missouri in 2015 or 2014, sorry. And then the federal government under President Obama came in with some federal funding to help assist in the purchase of those programs, and that they they they went like wildfire. Because they were really seen body cameras were seen as a panacea in some ways. Like the theory of change there was if we just monitor officers' behavior, they will use less force. I never thought that was, I mean, actually, that is the question that brought me into academia. As a cop, uh I was involved in a shooting, one of the first, if not the first, that was captured on body camera. And and um it was shortly after that that sort of the national fever took off around these technologies. And I heard very smart people, who some of whom I consider colleagues and friends now, Mike White, for example, um talking about cameras as a re a potential reduction for use of force. And to their credit, that generation of scholars, um Barack Ariel at Cambridge, Mike White at ASU, a bunch of others, Cynthia Lum at um George Mason, like the field came together to put together a pretty impressive research agenda to test that theory. And in the end, found that it didn't really affect use of force rates, right? Um, after 80 some odd RCTs, if we go by uh Dr. Lum's uh kind of uh synthesis of that literature, um she she and her colleagues uh uh end up saying, yeah, it doesn't, it looks like the the use case for for use of force was probably overstated. Um but that doesn't mean that technology number one isn't being used widely, right? Like it's one of the things about uh technology is once it's adopted, it's kind of hard to put down. Um and two, doesn't mean that it doesn't have other potential uses and effects. And so I think AI, the age of AI that we're entering now, um a lot of it is based around the body camera, interestingly. And so it gives us some new opportunities to test for those effects.

SPEAKER_01 14:46

Right. And that's what my next question was going to be. You touched on this a little bit earlier, but I I didn't want to pass by the opportunity for to talk about AI and the claims with respect to the application in policing. And there are a lot of other new technology, uh tech programs that have been marketed to the police, extensively marketed to the police. Uh I've run into an issue with um uh uh things such as like e-citations, patrol finder, and that the it's just absolutely flooding the police. But before we get to the question, I do want to mention something that you said that it did it it does influence your behavior. When I worked in Houston, uh we're talking in the 80s, in 85, they were finally putting computers in the cars. Now these were very rudimentary computers, but even my partner, Mike Wisnowski, he recognized the fact that what we were doing is we were patrolling the main streets more at like two in the morning because that's where the cars were, so we could plug in the license plates to see if they were wanted. He and he mentioned, you know something? We're not patrolling the neighborhood anymore. And it was like, holy shit, you know, that's that's right.

SPEAKER_00 15:44

We had a huge I was just talking with some cops about this, um, the modern version of what you're discussing, which is so when I I I've been out on some ride-alongs, and one of the things I noticed, so when I was an officer, if you say you wanted to find stolen plates, you had some time, you had some downtimes, you want to go find some cars, stolen cars. You're gonna how do you do that? Well, you drive around behind cars and type their license plate into your MVT or your laptop. Um, this forces you out onto the road. You have a little bit more, you have presence in the community, uh, visibility, etc. What I saw from these young patrol officers is they weren't doing that. And the reason was uh they had license, I'm not gonna mention any particular um companies, but like they had automated license plate reader systems in the city.

SPEAKER_01 16:33

Okay.

SPEAKER_00 16:34

And so the city is ringed with these cameras. And what the and and they also had um acoustic, I won't mention a company name, they had acoustic uh shot detectors in the city. And what the patrol officers were doing, it makes total sense if you think about it from their perspective. They're like they're sitting congregating at uh let's call it a uh a convenience store near the center of the city, and they're waiting for their phones to alert. They're waiting for their phones because when a stolen plate hits one of those ALPRs, it alerts immediately to their phones about direction, location, and car description. And then boom, now they're off, they're gonna go get that car. Um, or they're getting alerts from the shot detection system. So I I do not want to qualify this or being being heard to qualify this as bad. I don't think it is. I think it's just different than how Scott was patrolling Houston in the 80s, Ian was patrolling West Jordan in the 20s, early 2010s, and how cops are patrolling today have different factors at play in in how they're going to be moved around that city. And that has obvious implications for visibility, uh, reaction times, downtime, proactive policing versus reactive policing, has it's fascinating uh time to be studying policing because I think a lot of shifts are underway.

SPEAKER_01 17:54

Oh, I I couldn't agree more. And the the the availability of data of different kinds of different sources, and I'm still I'm still doing surveys of police officers. I just can't get enough of getting their opinions on things. But then the idea of having you know shot detectors, the the the uh the the body camera images, uh the data is just there's tons of it. So let's let's let's get into your study uh and and now uh this is something we usually gloss over a little bit is the uh the basic research question, which we'd like to know and how you did it. But normally, as I say, we gloss over, I I don't want people to to lose interest in the conversation, but this is something that the listeners, particularly the police chiefs and the upper management, might want to know is okay, what were you studying? How did you measure professionalism? How did you define it? And you mentioned the self-assessment versus uh supervisor mediation for versus treatment groups. So go into this and talk about this in a way that you're talking directly to a police chief.

SPEAKER_00 18:49

Yeah, so we're what we're interested in, the research question is can we change officer behavior by giving them near real-time feedback on what their own behavior was, right? Um officers don't typically get real-time feedback on their behavior, right? Like it's policing is often a solo or at most a two-person uh sport. Right. So uh for the most part, it's you interacting with someone uh for a short period of time and then you leave. And like most of them are not gonna complain, most of them are not gonna write a letter of commendation, you're just never gonna hear from that person again. So there's a real break in the feedback. So the theory is like by using body cameras and by automatically auditing them using AI, we can get some measures out of that. For example, in our study, we're very we're we're primarily interested in professionalism. Professionalism in itself isn't doesn't get measured, like it's not like the body camera hears professionalism. We need some sort of algorithm or way of thinking about that measure. So how this particular product works and how it thinks about professionalism is every interaction begins as a standard interaction. And then it can either become highly professional or substandard. All right. How it becomes substandard is if it picks up in the language of that body camera the officer using directed profanity, threats, and insults. Okay? So hey, you nympkum ninkum poop, stop it, right? Like that would be insulting language. I I'm I don't think anybody uses it anymore, but that would be considered sort of insulting language. You can imagine the range of directed profanity. If your imagination fails you, please see one of my previous studies called Fuck the Police. In Police Quarterly, it has table one, 50 uses of the worst fuck used in policing, and let your imagination run free.

SPEAKER_01 20:49

Yeah, you already we you already did a podcast with I think with Jerry Radcliffe about that one. So that stopped me my tracks. I couldn't talk about that. Okay, continue. Yeah. Yeah, I've said that the self-promotion. I I said that to a you know to a class of students. You know, walking into a loud music disturbance or something like that saying, everybody shut the fuck up is not the way to start the conversation.

SPEAKER_00 21:08

Probably not. Right. And I don't know anybody in policing that thinks it is. There is, I think ever most reasonable thinkers in policing think there's some room for profanity.

SPEAKER_01 21:16

Um, there's a social Sykes and Brent article from back in 1980 when uh they meant they mentioned the use of progressive to to to like you finally had it with, hey, sir, stand on the sidewalk, I'm doing this thing. You know, and and by the fourth time, it's like, get your fucking ass on the sidewalk. It's actually a tool. But again, we're digressing. I apologize.

SPEAKER_00 21:37

And going all the way back to Van Mann's the asshole, right? Like we there's a there's a there's a my only point is like there's a there may be a place where I don't want to be heard as saying like anti that guy doesn't like swearing. I I like it enough that I wrote a couple articles on it. So um but I don't know any reasonable observer of policing that says like you you know go to that music disturbance and say, Turn the music the fuck down, or on a traffic stop, like give me your fucking license. And we unfortunately see this type of behavior, and nobody really observes it and says, Yeah, that's a highly professional encounter. On the other hand, how does one measure highly professional? How should we distinguish between kind of day-to-day, good standard policing and highly professional policing? And I would turn your attention to maybe a motor officer. When a motor officer stops somebody, you know, they're they're they're primarily engaged in traffic enforcement, and they tend to have some sort of script that's going to go along with what they're doing day in, day out, multiple times a day. Hey, you know, I'm I'm Officer Smith with the Plainview Police Department. The reason I stopped you is there's a bunch of explanatory language that's kind of coming before at the very beginning of the encounter. And then when they bring the ticket back, because they're a motor cop, so you're definitely getting a ticket, they're gonna they're gonna explain like why road safety is important, why their agency focuses on it and what hopes that they are able to have a better day. Um I would say that there is one reason that we think of that as like a highly professional encounter, is the bulk of explanatory language. And that's exactly what we measured in this study as well. So not just in traffic stops, but across the breadth and depth of police activity, our officers, you know, before you do that frisk, before you make that arrest, before you write that citation, are you, when possible, using explanation so that the person on the other side of that encounter understands why you're doing what you're doing? This is, it might sound a little bit vaguely like procedural justice. I want to emphasize this is not a procedural justice paper. Um, we don't use measures of procedural justice, but there's an element to it where in order to get better uh cooperation just day to day in law enforcement, we tend to explain ourselves. And that's nothing new. That's not something I invented or our team invented or the company invented. I think that that's boiled into professional policing, right? We understand that we're going to be doing a lot of explanation. The the voice is an officer's primary tool that they're going to use every single day on the job. That is how we get police work done. Um, and so it's important to measure. So that's how that's what we do to take that long explanation. We're going to measure over 200,000 videos over six months in both in two different cities, one police department, one sheriff's office. We're going to extract transcripts from every single one of those videos. We're going to run natural language programming over it. And for every encounter of those two hundred nearly 200,000 encounters or videos, we're going to develop a professionalism score. And that's just standard, high or low. And we're going to deliver that directly back to the officer, either directly to that officer or through their sergeant or first line supervisor. Um, and this has important implications for like how we think agencies ought to be managed, or like how how can an agency that's looking to get improvements um what are the different ways we could consider that might work?

SPEAKER_01 25:24

Right. Real quick, uh, before we get to what were your findings, I'm suddenly reminded of, and I I didn't care about this. When it was on national news, Tony Romo getting ex uh arrested for for DWI. I said to my wife, this is my wife, this is not a natural story. I don't care. I I genuinely didn't. But then they showed the video and it and the officer was polite. Uh you you could catch tone uh as well. And I'm not sure if if AI can catch tone of voice rather than just in in with with words. But I thought to myself, wow, that was a really a good professional approach. She was polite, she was instructing, she was tell answering questions when he had a question about something. So all that all that being said, officers were then either showing it themselves or the supervisor gave it to them. Did did this improve their behavior down the line?

SPEAKER_00 26:14

Yeah, there's yeah, I you baked a couple things in there that I want to address. So tone possible, but not done in this study. So this study is really just running off of the text. Number two, Tony Romo was uh uh a member of the Dallas Cowboys, and so we should be happy with anything bad that happens to them. Although I'm happy that the uh uh officer used professional language. And three, what were the effects in this study? Um it's nuanced, and I'm glad we're we have the time to talk about it. The top line is we saw improvements to police professionalism behavior and behavior in both sites. Those those effects were statistically significant and in the direction that you might want, right? Um, the the nuance lies beneath as it always does, and so um it's probably easiest to talk about that in terms of the two places, the two agencies that we situated this study in. However, you want. The first, so the first agency was Aurora Police Department in Colorado. This is a police department, um, as opposed to a sheriff's uh department, which might also have uh implications. But the Aurora Police Department, if you don't know much about it, has a long-storied history. Um these there are officers at this agency who have responded to 1999 Columbine, as well as the um the Batman uh movie shooter mass shooting uh that you may have known about. So this is this is an agency that that's seen some things. Um it's also an agency with a lot of media attention because of during recent years uh the the death of a young black man in which following which several officers and several EMTs were charged with uh various crimes related to his death. So uh it's also an agency that was under state level consent decree. So not it wasn't the feds coming in, it was the state of Colorado had placed this agency under consent decree. So that's sort of um a little bit of context. In that agency, we did not see improvements to via an increase in highly professional behavior, but we saw the improvement through a reduction of unprofessional behavior or substandard behavior. And this effect was most pronounced coming when when the information came through their supervisor as opposed to being self-directed. And so you might ask yourself, like, why? Let's just hold that question for a minute while I explain what happened in Richland. In Richland, at the base rate in the control group, we just don't have a lot of unprofessional behavior. This is a sheriff's department run by a very long-serving Southern sheriff, uh, Sheriff Leon Lott, a former president, recent president of um the National Sheriff's Association. He is a very media forward sheriff. He um there are several Netflix shows, documentaries at the agency that you may have seen. There are several sort of on patrol and live PD type um programs at the agency. And I say that to sort of differentiate it in terms of if you're an officer at one of these two agencies, think about how different your experience is in life. If the media is always around and sort of promoting a positive view of your agency, as opposed to you're an agency in an in a in an agency that's under consent decree, a lot of media pressure, and there's not a lot of positive media stories. When you see an additional layer of supervision or transparency coming, you may not see it as as in as a positive light. Um so in Richland, we saw they're not opposite, but they are interesting comparisons as to how the outcome came out. There, we don't see a lot of unprofessional behavior in the first place, and we don't see a reduction in it. So there's not a significant reduction in that unprofessional behavior. But there was a nearly 80% increase in the highly professional behavior. So we saw the ceiling on professionalism move in Richland and the floor on professionalism move in Aurora. Here's where I want to pause and have a little bit of humility for the scientific, what the science can say. This is was this was an experiment. It's an RCT. We we consider it the highest level, sometimes you'll hear it referred to as gold standard. Not only was it a randomized control trial, it was pre-registered, meaning before we ever saw data, before we ever ran an analysis, we wrote down on paper and committed exactly how we're going to go through this study. In fact, this is the first and to this date only pre-registered RCT to ever appear in our discipline's flagship journal of criminology. Um so I'm proud of the way that that we can say this was done at the highest levels of scientific execution possible. But what an experiment doesn't tell you is why, right? And everybody wants to know the question, the answer to that question. Why did professionalism go up over here? And why did uh bad prof or substandard professionalism decrease over here? And the the reality is we don't know. We can't know. All we can do is guess. So what follows are guesses, which is fine. They're educated guesses, but they are guesses.

SPEAKER_01 31:46

I've been there before, yeah.

SPEAKER_00 31:48

Yeah, it's just it's it's something people outside the the experimental world may not realize. We're not very good at the why. We're good at causes, what happened. Um the why is is sometimes a challenge. Not sometimes it is a challenge. Um because you're not varying your department, right? You're you're you have no control, you can't randomize it. Um I I think there's something to one of the oldest sort of social science truths, and that is when we know we are being observed, our behavior changes. And we and the there's a specific way in which that behavior tends to change. It's that we we we tend to reduce behavior that we know we shouldn't be doing. So think about the Aurora case. It's not the case that Aurora officers were unaware that they shouldn't be using directed profanity and insulting language before I arrived, right? It's not like I went in there and lectured them, like this is what you shouldn't be doing. They already knew that. And they what what else do they know is they know that there's this program coming in called Trulio that's going to be watching that behavior and reporting on that behavior. In fact, it's gonna tell them when it gets flagged, right? It's gonna come directly to their inbox and say, you use this language, don't do it. And so if you're not highly engaged with the technology, which I think was is fair to say they weren't, and in my conversations with Aurora officers, there was a great deal of resentment. Um, they saw it as another tool of suppression, right, or investigation. Yeah. Um, they were quite, some of them were quite angry with me and and the other researchers because they sort of saw us as the technology itself or representatives of the technology itself, even though we were totally separate. Um and so, in that, if you if you you know, if you give those officers some grace and think about their experiences, I think that that's a pretty reasonable view. Like everybody is coming in to investigate your agency and you, and this looks a lot like one of those tools, right? It's gonna look at all your words and give you scores. And so, in that environment, we didn't see that big increase in highly professional behavior, but we did see a decrease in in what they knew they already shouldn't be doing. Um, and that's an old social science truism almost at this point, that um perhaps that's the reason that we saw this these differential effects. On the other hand, I want to tell you a story about Richland. When we went and interviewed at Richland officers about their experience, um one story, two couple stories really stood out, but I'll just tell you one of them in the interest of time. The officer, I one officer kept talking about how they really liked getting these um highly professional encounter ratings. And I said, well, you know, and then he described these like, and especially once we figured out that all we had to do was use a lot of explanation. And that was very interesting because we didn't tell these officers how professionalism was being measured. We didn't go in and say, use 25 or more words of explanation and you'll get this highly professional behavior. Um, and he didn't say 25 words, but he said, you know, once once we figured out that all you had to do is explain, um, and that gets you highly professional, uh, it became a lot of fun getting those those highly professional ratings. And I said, Well, tell me more about that. Like, how did you uh what led you to that belief? And he told a story that early on in the experiment, they noticed, they being the patrol guys, noticed that their motor officer friends were telling them that they were getting a lot of highly professional encounter ratings. And these patrol officers figured out real quick that all they had to do was come up with a script sort of of their own to adopt into other patrol activities. And so they basically modeled themselves after motor officers, which I think is a fascinating like gamification sort of story, potentially, about how to improve officer professionalism. Ultimately, there was a bunch of cops competing at some level for those ratings. And like if you can take advantage of, if if I would say to a chief, if you think your agency might benefit from more highly professional encounters out in the community, and the way we get there is by emphasizing the competitive nature of officers, which I think is true. Like officers come with a great deal of competition uh built in, then I see that as a win-win.

SPEAKER_01 36:36

That's that's fascinating that well for first a couple things. You get you do see improvement, they're not as bad as they were, they're better. Okay, which is great. So as you say, the the the the the floor has raised and said it that way. And then the deal in enrichment is like they're they're already doing a good job, and it's are hard to improve upon already pretty good. And and again, not the best analogy, but if I'm a B student, yeah, an A is nice, but B is not bad. Now I'm gonna go with B pro B plus, okay, great, not E minus. But if I'm moving from like a D plus or C minus to at least a a C or a B minus, that's pretty good too. So you're getting better in both ways. But this thing you're talking about here, this this idea of and here's where I'm gonna push back a little on you, isn't really all that more professional. If you're learning that all you have to do is have a script, and I I like that idea, don't get me wrong, that uh I'm assuming I'm assuming motor officers are basically out there on the highway stopping traffic, and that's basically all they do. Am I right about that?

SPEAKER_00 37:38

Not necessarily the highway, but yeah, but they they're they're traditionally going to be traffic enforcement and traffic accidents.

SPEAKER_01 37:45

Versus a patrol cop like I was that you know answered calls for service, domestic violence, bar bar complaints, minor traffic accidents, okay. So you're right. They they probably have a script that gets them through each of these traffic stops in an efficient, polite manner. There's no sense in in getting into an argument with this person you're gonna give a ticket to, it's just gonna frustrate yourself and you're gonna go home and be pissed pissed off at yourself and because they're pissed off at you. So I'm still wondering I wonder if it's professional if that's all you're doing is is mimicking something that you know is. It's fascinating.

SPEAKER_00 38:16

This this came up um in the study, it came up in with with um reviewers and friendly readers who who brought up the same point. Like, is this did we actually change hearts and minds? And my answer was I don't fucking care. Right?

SPEAKER_01 38:33

Right, I don't care.

SPEAKER_00 38:34

Better is better. I don't care. I can't uh consider me an experimental monkey, Scott. I don't care because I can't measure their hearts. I can't measure some subjective sense of I could, but I just maybe won't trust it, let's put it that way. Whether I've shifted somebody's core policy, moral, ethical beliefs, something like that. What I care a lot about is behavior, because that's what's seen by the public. Um and I care a lot about it in that that's what we can plausibly reach. Um anybody can tell me that they are deeply committed to the ideals of procedural justice. Tells me nothing about whether they are engaged in procedurally just behavior. Uh somebody can use cheap words without engaging in expensive behavior. Um, behavior is expensive and it's hard to change. Beliefs are easy to mimic and hard to measure. So, from an experimental point of view, I care a lot about the behavior. And um, I think our study at least points the way forward for a relatively straightforward approach that doesn't rely on traditional mechanics of the sort of pseudo-militaristic nature of policing. In other words, it doesn't require a new IA sergeant. It doesn't require expensive investigations, it doesn't require anything like that. The strata upon which, or the feedstock, I called it earlier, the data that we want is already there, present in the agency. And what we're talking about is implementing a layer on top of it and feeding that information back to the officers themselves. Remember, neither good nor bad behavior in this study was um for for our self-mediated officers, was reachable by the agency at all. No supervisor saw it, nobody, there was no supervisor to give them a pat on the back or uh a smack on the ass, right? Like there's no carrot or stick other than the internal motivations of those officers themselves. And so um, yeah, behavior is expensive and talk is cheap, and we should focus on the former.

SPEAKER_01 40:52

Based upon what you did, what you found, what are two or three implications for the police that might be listening to this conversation?

SPEAKER_00 41:00

Well, number one, engage in evaluate careful evaluation before adoption. Um set up good tests of what you're spending money on. Um I know of agencies right now that are facing contracts from the largest vendors in this uh in the the like police technology space that are eating upwards of 25-30 percent of their yearly spendable budget, right? Budgets in in policing, as you know, are 95 to 96 percent of any budget is eaten up by personnel costs alone. And so like the spendable budget is much, much, much smaller than what you might see on a city council budget. Um, so chiefs have to make hard decisions, and there's a lot of opportunity costs when we commit to multi-year technology contracts without any proven effect. Um number two, don't think of police behavior as only reachable through agency discipline or rewards. Um, officers are professionals, the vast bulk of them are professionals, meaning they want to get better at their job. And one of, but but we don't do a great job in policing at providing tools, feedback tools to those officers so that they can learn in real time. I mean, it's one thing to sort of you know go to an incident, make a mistake, it goes into IAA for six months, it comes back out, your FOP representative gets involved, there's six more months of hearing, and finally there's a written reprimand 18 months after the incident. What did we learn, right? Like what did we actually change in terms of behavior? We we probably did a good job of um laying in uh the liability protections that we need, and I'm not making small of that, that that's an important role for the agency to do. But did we did we change individual behavior with that? I I'm not convinced, right? Um so by spending some money on layers of feedback that go directly to the officer and provide them at least with the opportunity to uh learn and and become a better professional, maybe we maybe we um speed up that flywheel of experience. Maybe we don't need to wait 13 or 15 years for someone to become the the best version of an officer that that that they can be. Um yeah, I those are two good implications.

SPEAKER_01 43:28

Excellent conversation. Uh thanks, Ian. I appreciate your time.

SPEAKER_00 43:32

Thanks, sir.

SPEAKER_01 43:32

All right, have a great day.