Model Risk Management & Charlie Brown with Chase Ogden

In episode 9, the guys are joined by DCG Quantitative Consultant Chase Ogden. The trio investigates a panoply of subjects in the MRM space, including the importance of building out a data strategy to promote better decision-making, thoughts from the CECL frontlines, and why there is always room for bankers to better utilize quantitative results ("with humility”).

For more insights and ideas, visit DCG at DarlingConsulting.com or follow us on LinkedIn.

Transcript

On the Balance Sheet: S3 E9: Model Risk Management & Charlie Brown with Chase Ogden

Transcript 

[Vinny, 00:07] 

Welcome to On the Balance Sheet season 3 episode 9. Today, we've got a great episode lined up. We've got Chase Ogden from DCG. Chase, as everyone's going to find out within the next several moments, is really the Jim Nantz of DCG. He's got some tremendous pipes. I want to give a very quick story about Chase before we get the interview going here. At our 40th annual Balance Sheet Conference, we did something a little bit different this year. We had a large panel, with a number of our colleagues up on the stage answering questions and, for some of us, I happen to be one of the people up there, was can be kind of intimidating when you're sitting on a stage and a few hundred people are out looking at you and there are bright lights in your face, it's a it's not a natural sort of scenario and yet we had one guy who just stole the day and it was Chase Ogden.  Chase, thank you so much for joining us. It's a pleasure to have you finally and to learn more about what you do here at DC. 

[Chase, 01:01] 

Happy to be here. Thank you. 

[Zach, 01:02] 

Chase, we're really excited for this episode and for the listeners, it will be a little different episode. We're going to focus a little more on model risk management in that area. Chase is a quantitative consultant here at DCG. He's got a pretty vast background in the banking industry, and he's been working with us for about 5 years now. So, before we get into it, Chase would you mind just giving the listeners a little bit of a background where you started in banking and how you got here and kind of what you've been up to? And then we'll definitely dive into all of the acronyms of the MRM model risk management world. 

[Chase, 01:31] 

Absolutely, that's a fair place to start. I got my start in banking because I followed my wife out of school. So that's, I guess, an interesting way into financial services. It was not through really any desire of my own. We're both statisticians by training, and we're in school together at the University of Alabama. And they had frequent recruiting events where large name industries will come in and say, hey, we're looking for applied statisticians, we want to meet with your department and we were both in the PhD program at the time. And JP Morgan sent some people down and they talked to Jenny and they just absolutely loved her and said, you know, you've got to come to Ohio. And so she took a trip to Ohio, and they made her an offer she couldn't refuse. So, she said, I'm leaving. I've been, I said, but wait, wait, don't leave me behind. So, I followed her to Ohio, and, you know, they're really interesting jobs in in Columbus, but I I got a job with a regional bank there and really haven't looked back. I had the benefit of instead of starting my first job and then through every job I've had since I've sat close enough to executives that it's made the work way more interesting, you know, having technical skills and applied statistics is interesting, right? Academically, you can do lots of different things with it as the call it air quote. What do you do with that? Well, you can do a lot of different things. And your knowledge of the industry or the application is what sets you apart, and you can't get that kind of skill in which to apply your techniques or your applications, unless you have access to somebody that's really going to take the time to say what is it that we do here? And so, I got my start in, in retail banking there and in retail administration, which is, you know, a fascinating place to work. It's the intersection of the bank, being a storefront business but also being the engine that runs the rest of the machine. So that's a cool place in my mind. Now that I know what I know now, I think that's a really interesting way to get started in banking because it's not, it's not a traditional finance approach, it's it's much more boots on the ground and the people that are part of our organizations. So, from there, I leaned more heavily into some of my quantitative background and and worked for other mid-size institutions and in the defense space zone, so capital planning and then into treasury. And then spent more time on data and analytics more generally. So, building a department for how do you wrangle enterprise data? And how do you get practitioners effectively interfacing with IT and in a way that that's meaningful and you know that's a broad swath of what I've been able to do in my brief stint in, in the industry. But I've been able to see a lot and and learn a lot and I think there's always a place for a quantitative result, with humility, right, it's useful to have data to support decisions. It's what we do here at DCG all the time. That is our place, but again, with humility is the importance, because there's so much more to the story than just what the numbers have been and and what can we do with it? How do we interpret it, and how do we use it to make the best decisions we can? 

[Zach, 04:51] 

It is really well said and Chase, one of your jobs I know you were director of data strategy and analytics at one of those banks. Is that what you're referring to at the end there in terms

of trying to to bring that quantitative piece to the IT space, but also to some of the decision making at the at the bank? 

[Chase, 05:04] 

That’s right, so how do you get organized around all the different activities that it takes to use data? So that's upstream, that's how are we gathering data, holding data? But then how do we translate that into a way that it's usable to the rest of the organization? The simple example, the trivial, trivial example that everyone's run into if you've worked at a financial institution is, oh well, you know someone runs that report, and quite often prints it and hands it to me and right that that is the caveman version of what what we were trying to do there is you don't need somebody to push the button to print the thing to give you a binder filled with paper that, I mean, what's on the paper? And was it the information you needed? Did you use it to make a decision? I mean, it's one of those things that when you actually engage with the executive management team on that you say show me the binder that you live by every day, show that to me and flip through it with and tell me what it is that you do with this every day, or every week, or every month, and how that's informing your decision to make this a better organization. I found very quickly it was alright this front page, there's one number here that I always look at and then I flip 5-6 pages to get to the next number that I actually look at, but in between the first page, one number and the 7th page, one number are just a wealth of information that it's either not presented in the right way. It may be misunderstood. It may not be, it may not be useful. It may be that it's gathered in such a way that, yeah, this would be good, but I know that our system is gathering this and recording it the wrong way. So, what I have at my fingertips could be good, but isn't, so that that to me is the sticking point. It's why data strategy or data analytics or any other buzzword you want to put around that that's critical. It's how we turn the information that we're gathering at our organization into something meaningful that we're making decisions off of. 

[Vinny, 07:11] 

It's a very interesting sort of storyline. I kind of think back to what I've been told about how George Darling actually founded DCG. I think he was in software sales and then recognized that, you know, we were basically selling what was sort of the antecedent to what we're doing today. And you recognize that these leaders at these banks had no clue what to do with the information. And it wasn't usable, and he recognized that there's a space in the industry for people to go out and interpret that data and make it usable. Of course, that was, there wasn't nearly as much available back then that there is today, but clearly there is and, and you're right, so much of what we do today is driven by the data we, you know, we're not going to come up with some strategy based upon gut feel, that doesn't work. Maybe it did for a lot of institutions for a very long period of time, that's obviously no longer the case. So, thank you so much for for kind of giving that overview. I guess we'll start out if it makes sense. So many people that listen to this may or may not have a real strong understanding of CECL. CECL is something that's really kind of changed the calculus in our industry, and I'm kind of wondering if you go back and look at the prior way that banks sort of calculated their expected losses they used the incurred loss method, you know, FASB got a hold of it and has changed it and now we've got this cool acronym called CECL. FASB probably pretty good at creating perpetual job security for the for accountants, so that's, that makes sense that they would have done something like that, but from my understanding at a very high level it was done so that institutions or the investors and folks could have a better understanding of the information in the financial statements. I guess this is kind of a loaded question as we stand here today a few years into this, what is your reaction to that, that initial goal of FASB's migration to CECL? 

[Chase, 09:05] 

That was a loaded question; I think, though, that the intention is good. I think we can be critical of the way that it's been implemented, but the idea that management teams should be free to use all of the information at their fingertips to estimate the reserve, I think that's the right way to go about it. So, for ALLL that was not the case. It was, it was some math on a few years looking backward in time and then deciding that, you know, that's not enough -we should probably make that number bigger and do a little bit more math to make that number bigger. But as we think about users of financial statements, how do you interpret that if what you're trying to understand is the credit risk that management has on the portfolio today, looking back at some math that you would have access to otherwise and you'd have no insight into why that number is bigger than what it had been historically, it would be really impossible to judge, you know, is is if all I had was an analysis of your financial statements, I really would have no idea what's going on with the portfolio. I'd have to make guesses at a portfolio level as to what may be going on and then infer from there. But if we give FASB credit for changing the game and allowing the models to look forward, I think that actually does, and in theory, provide more insight into the utility of that number. So, if my reserve is now 100 basis points, and I know what management is viewing as current conditions and expected conditions through the remaining life of the portfolio. OK, well, that's different than just having it's been 50 basis points historically, but for some reason management has 100 basis points of reserve on the books was what does that mean? While I see exactly why it was 50, I don't know why it's 100, and I have no idea what calculations went into that. But knowing now that the model must use all this other information, that 100 basis points estimate has a little bit more meaning to it.

[Vinny, 11:23] 

Yeah. So how are these models evolving now? I mean, maybe I should take a step back. I mean, we're a little bit into this whole CECL process. Some folks sort of outsource that others kind of took it on themselves. How would you sort of characterize the early stages of it, the adoption where institutions are today, what were some of the the pitfalls, if you will, if if you did outsource it, or even if you took it on your own and and what are we kind of finding out here? Which I think is just natural that folks, as we transition, we're kind of going to run into some things, so just kind of curious how your perspective is. 

[Chase, 12:00] 

So, management teams had a wide variety of choices at their disposal for adoption, and for public filers this was a 2018-2019 decision. For smaller community institutions that was delayed for a long time. Many hoped, perpetually perpetuity, that that turned out not to be the case. This was unavoidable, but the larger institutions that went first were, I think, many wanted some kind of security. I want, this is a big change, I want to make sure that what I do is accepted, so I want my my audit firm, I want my my external accountants, I want them to know early and often what it is that I'm doing. I want them to know who I'm partnered with. I want that partner to lend credibility to my process. And I want this to be a non-event when I actually have to make these new entries and and adopt the new standards. So, I think that line of thinking was probably predominant for for most institutions - it was to get the security to make the to make your adoption you you went with a vendor. We where I think otherwise I think firms, when left to their own devices to understand the way the models work today, that they've actually adopted and what they're doing right now and they're X number of years into it, even for the community institutions, they're only one slightly more than one year into publishing these results.  I honestly think they would look back on it and say, you know, I I probably would have done this in-house and the level of complexity that we see is not all that much different than what they're doing with other models internally. And they're if they are using, call it, a more sophisticated approach, if they do have a discounted cash flow model somewhere in there that's well understood right there, there are a lot of ways, you know, the the accounting standard itself was prescriptive. If it didn't define what management teams were required to do and where vendors and and other firms have landed in the space is that you and I could sit here with you guys. We could pull Excel out and we could make a compliant model in an hour and it would be credible and give us a day or a week and we could do something that's credible and fancy and wow, OK, then then did we did we need a partner for that? Which is not to disparage the need for the partner. There's their capabilities that a lot of firms would have avoided or never gotten, so access to economic forecasts or horizontal perspectives on how to tune the assumptions as you use the model over time. Those I don't think can be overlooked, because if you're if you're just trying to do everything in-house and you're you're myopic in that view, you'll never use the model to its full potential. You'll just be allowing everything to stay static as much as possible, and then to the extent you feel the need to move the result, you'll do it qualitatively with with very little support, which is not a a good place to be. You certainly don't want want a more complex model like this to be dependent upon fluctuations in the non-modeled portion, if you get my drift. 

[Zach, 15:19] 

Chase, is there like an asset size in your explanation there and some of the differences that that you're seeing in terms of, hey, it's ten billion above can do a lot of this in-house for smaller banks are probably less likely. Or is it no, it's it's, you know, what you said was pretty much ubiquitous across the asset class, it just depends on the people that you’re working with. 

[Chase, 15:40] 

I actually, the latter; I think it greatly depends upon the people. I think if you trust that we have a strong internal team which I've come across quite a few at large institutions that are call it 30 to 100 billion, and I've come across many in the community space of call it, we're the 1 to $5 billion institution, but we have a couple of folks in finance and accounting or we have someone in credit and they're they're capable and can build this new process for us and it it leverages capability they had in the old world, but extends it to be compliant in the new world and they fully understand it and that's a place where they want to live, then that's a good place to be. 

[Zach, 16:21] 

Chase, I was going to segue a little bit before we get into some of the model risk management stuff. I did want to kind of piece a credit discussion here because I know you can't talk CECL without talking about credit. And for the listeners, Chase just came down here to to our studio and said he was just talking to our analyst group. Give them an update on what he's seeing from the credit side, so could you, just you know, whether it's in the kind of lens of CECL or or other things that you're seeing, whether it's on stress testing, capital stress testing, credit stress testing, what are you seeing out there? What are banks doing like from that perspective? 

[Chase,16:53] 

Well, that's that's good for the for the analyst group in in particular, they were interested in getting, you know, we we I think have started at a place that assumes that the listener knows what CECL is and what's going on here with the analyst group. They don't interface with such models all the time, so with them it was starting with, well tell me what a balance sheet reserve is for and where does it exist in your organization and and why do you have it? So, we talked a little bit about that and giving them grounding. My team goes and does trainings periodically in the industry to teach new, call it three months to one year in the industry to show them, you know, if you were analyzing a bank or you were part of the bank here the different functions, here are the different risks that are being managed, the credit risk, of course, being one of them. And we very quickly migrated to, how do we understand the reserve and why is it useful in interpreting the results of the bank and assessing its credit risk? So then going back to your earlier point of, is it more useful now? Yeah, I think so. And as I presented to them, it's really in the sense of ongoing performance monitoring is the way we think about it. The the way to interpret those results between institutions in a way that's meaningful, and we spend a lot of time as a team validating other models or or other other firms, CECL models that is, and seeing what they're doing to get comfortable with the result that's coming out of their model. And it's surprisingly very little, right, you know, talking about going with your gu and and trusting a decision. Well, the model hasn't moved very much, and we felt good about it at adoption so we feel good about it today, should you? Should it actually have been moving this whole time and it hasn't, and it's not working as it should. Do you have enough insight to know what that what that should have been doing and and is it doing what it's supposed to be doing? So, for them I I gave them a few examples of, you know, as we've talked, if I wanted to have a forward-looking component what would that look like? My example for them was in was Charlie Brown. I'm a big peanuts fan, and he's a cartoon of him standing there and saying, I have a a feeling of impending doom. Well, at the 8-9 and 10 crisis, if you're in 2007, would you have had a sense of impending doom at that point? 

[Vinny, 19:12] 

No, not at that point. 

[Chase, 19:14] 

How about early 2008, right? So, there was a, there was a point, though. My point is that there was a point where you said I was feeling OK when I think about the future. And then there was a a point where you say I'm not feeling okay, and the way you would want to measure and monitor and protect against credit losses is as soon as I have that that moment, I want to be able to take action on it. The old regime wouldn't allow you to do that, and having the ability and the new model to be able to say I have a feeling and I want to act on that feeling and support it and have that be a part of my financial statements is a a key difference. Now in hindsight, absolutely we would have seen reserve build, that whenever

management teams had that, that first inkling of we need to set something aside this is not going according to plan and something may go sideways. Easy to see it there, less easy to see in COVID. So that's an interesting example of oh, what should have been, I guess, from a negative perspective because that would have been a significant losses, but it should have resulted in a credit cycle, but through intervention of the government, transfer payments to households and call it relief for business borrowers resulted in a non-event and, in some cases, consumers using their transfer payment to pay down debt which made them look like a better borrower than they were going into what should have happened. So, if you consider what the reserve model should have done, they should have said, oh my goodness, this is going to be bad and you would have built a tremendous amount of reserve. And then use that right? That's the the idea. The mechanism would have been, hey, we built it when we really built the reserve, when we realized we needed to have it. We then used it for when it was necessary, and then we returned to some baseline state, the mechanics of how that played out, where it was not nearly so clean. And the reality was, model said we need to reserve way more than we would ever consider because it was since it's so much more sensitive than than many firms had tested and, before implementing, way more sensitive than they thought it would be. So, the result was we need a whole heck of a lot of reserves. Management said we can't do that. So, there was a a tampering of, well, how high should we allow the model to go? But as we all remember, 2020 pretty quickly it was realized that oh well, yeah, this this might not turn into anything. The government has stepped in by the end of the year, we've got early trials of vaccines that look like it will return to business as normal very quickly, and as we look at 2021 gosh, you could have gone on a nice vacation to to Disney in 2021 because they had a special and there weren't that many people there so that it very quickly resolved itself. But what did the model continue to say was happening? The model was continuing to say, and I I don't know, right forecasts of what was going to happen were still pretty pessimistic through the end of 2020 and most of the way through 2021, and perhaps rightfully so. But the way, what would impact the model is you're now holding on to a lot of reserves that will go unused, then management teams didn't necessarily understand, how do I release it if I think that’s what's going on here is a non-event that we're that we're pretty much through at this point, now what? And that was, I think, a really rough start to something that was brand new, to something that was complex. It was supposed to help in a time that looked like the Great Recession. The next credit cycle didn't look anything like the Great Recession, it was totally different, and the government intervention was so strong that it mollified any result that we would have seen from a loss perspective that would have justified economy bad, therefore, credit performance is bad. It it negated that so, so now what? We find that in practice getting ready for the next potential cycle management teams still don't know what to do, so if the next thing that comes around looks different, which it will, how will my model respond? Will it be too sensitive like it was at the beginning of COVID? Will it be not sensitive enough because the impact of COVID was offset by some external force, but the next time we may not have an external force. So, what is a practitioner to do when applying this model, and then how useful is it in the longer term? I think that that's a harder question to answer, so that that the in theory is great, but the in practice is it's very difficult to use, and I think jumping all the way back to the thought of should we have partnered with a vendor or not or done this on our own, at least there's wisdom in the pack. So, if you are a part of the vendor relationship and you're you're in the same boat as other people, you'll at least be thinking about things together, sharing that information and trying to apply it as best you can because it is quite difficult to do and is not going to get any easier anytime soon. 

[Vinny, 24:52] 

So, what are you seeing more specifically as you move forward because, of course, now the Fed had just moved rates down. Traditionally when rates are going down, that's because there's some slowing in the economy and there is, depending on what metrics you're looking at, but yet for most of the banks that we visit, you know you're not hearing anyone kind of throwing their arms up in the air and screaming about their credit. You know, losses have been de minimis at best for for many, and you see some signs in the credit card space and so forth, but nothing just yet for most of the banks and the types of collateral that they're lending on for us. So how is the practitioner supposed to sort of understand what the next, what this environment looks like? I mean, rates could go down a little bit, they could go down a lot. No one has a crystal ball. How do you get that right and what is and sort of what is the guidance that you would give or or what are you seeing other institutions do in their models? 

[Chase, 25:44] 

Some of the best practices I think that we see is getting granular in the way that you're using your model and, as much as possible, if you were doing very little, so I didn't know what to do. I chose a partner to implement this model. I'm using all of the canned assumptions, all of the canned data, it's it it is turned into a push button solution, right? There's there's so little going on now that I have control over, I just put in my own balances and hit the button, get a result. If that's the place where you're starting, it's impossible to know where to go from there because you don't have the knowledge to adjust those results quantitatively or qualitatively in a meaningful way, and that's troublesome. And we do run into that more often than I'd like or would hope, I guess. The hope of, I think FASB would be that you're using a lot of information quantitatively. So that as you say, well what's what does today look like? Well, some things look bad, some things look good. If you had enough information on balance, you would hope the quantitative model could interpret, well, which things are more important? Some portfolios are going to be more sensitive to call it an unemployment rate. Sure, if Chase doesn't have a job, Chase is probably not going to pay off his car, right? Those kind of connections are very direct but become less so when you're looking at businesses. So, if I have a small to medium business in my economy, maybe it's closer linked to an unemployment rate, but what if I have large businesses that I'm lending to, and then that's secured by real estate and then they're right. There's so many ways just to to extrapolate there and you you come up with credits that are well insulated from just a a single metric that may be giving you all right for people, the economy is not looking that great right now, but I'm not trying to model people. I'm actually trying to model the borrower, right? What what do I have in the portfolio right now that is a risk to me, and that's where the granularity comes in. I need to have an understanding of what's going on and the a good way to implement, that is that accounting doesn't own the model and most instances where we see this stronger view of trying to link together what management is using day-to-day to decision credits, or at least to update them through the review process, is if credit admin or finance is the owner of the CECL model. And if that's the case, then a lot of this reporting is handing gloves. So now I'm looking at well, what should the models say reserves are, but give me a sanity check, what's actually going on in our market with our borrowers? It can be helpful to have anecdotal information from your, call it, regional credit officers or chief lender or some some role that you have that would know the stories of things that are becoming problems. And can then try to link that to them, knowing enough about the model and saying, yeah, the model is going to miss that. Because here's the problem we're seeing, we've got an uptick of non-accruals in our CRE. Well I know that's because for us, it's not in office, it's in C stores because we have exposure in a particular part of our market, this is something we need to be thinking about. The model is going to be looking at an unemployment rate, it's going to be looking at real estate prices, it's going to be looking at overall economic growth. Well, those metrics are way too far removed, right? We're talking about the performance of individual properties that are going to cash flow that that doesn't matter, right? And an uptick of, call it a 10th of a point of unemployment, doesn't matter in the grand scheme of how the model is going to interpret the credit risk that you’re experiencing. So being able to see what's going on on the ground and then interpret that in a way that's in the model, that's the important thing to do and we see it done well at some places and in less well others. 

[Zach, 29:53] 

Chase I'd say the the word model, we've used that a bunch, you know, during the this episode. Can you just, you know, kind of talking about CECL, but moving over to maybe to model risk management, the MRM acronym, can you just give us a quick definition, you know for the layperson or maybe why should bankers care? Yeah, especially, maybe some

folks who are in the the finance function or the folks that then I might talk to more of them maybe aren't in the MRM space that we're talking ALCO, we're talking strategy, we're doing other things like why should bankers care about MRI? And what exactly is it? 

[Chase, 30:28] 

That's a good question, and model risk management is all about trust. That's, that's how I introduce it to the layperson. We're trying to make decisions all the time. We're trying to use whatever information we have at our disposal. Quite often these days it's data and how we're transforming that data into a way for us to be able to make decisions. There's inevitably a model in that process. And depending on how it's built, what that methodology is, who did it? Was it a software? Was it me using a piece of paper or a spreadsheet? You should be able to trust that the results that are coming out of that model should be useful and should be used to make a decision, and if you don't have that trust, why would you use the model? Why wouldn't you just guess? And it's interesting to see the number of institutions that are heavily reliant right models are ubiquitous. I mean, we've been saying the word a lot, because it's used a lot. If we have a proliferation of of data and technology at our disposal, well, how do you use that? And you use it by trying to synthesize it and using a model that allows you to get some result, right? Typically, it's a prediction of some kind or or an estimate that allows you to to take action, that's what you want to do. But the worst possible outcome would be to take that outcome, or to take an action when the outcome was wrong in the first place. So, if I told you that your interest rate risk model was broken, and you've had a discussion with the Treasury Group in ALCO for the last hour talking about how the strategy should be this because your asset sensitive today, and I told you, no, man, the model is wrong, you're not asset sensitive, your liability sensitive. Does that change the discussion? Absolutely, that does, right, that we wouldn't have said anything we said in the last hour if you told me the exact opposite was what was true for your particular portfolio. And that extends to any place where management teams are making decisions. So, market risk or credit risk or liquidity risk. And if we're, it's really interesting to see and places that you really wouldn't think that you should have model risk management. So, considering expenses that you may have a marketing team that's trying to target your next wave of customers, how are they doing that? They're probably going to a vendor and they're buying market information, they're buying information about the potential new customers, they're developing some kind of model that's going to inform the strategy that the the old school would be we mail them something. That's not how that works. It works on clicks now and how many eyes, and how many clicks do we get in front of these these targeted ads because that becomes quite expensive for us to just go out there and target the wrong people. So, it can be anywhere and everywhere in the organization. We're trying to make the right choice, and it is an expensive choice for us to be in a position that the results can't be

trusted. For me, that's the easiest sell on why I care. Well, I care because I'm trying to run the best business that I can for my stakeholders, my shareholders, for my market, my community and altruistic view. Right, we're here to serve the community; that's what most in most of the thousands of banks and and credit unions in the US are here to do is to serve their local community. And how do you do that while you have financial strength? By making the right decisions and preserving enough capital. And if you are making all those decisions without any sort of foundation, or at least a foundation that has cracks in it, then that's troublesome. And that's the hole that the model risk management is seeking to fill. And before we jump, I have to say it's the the number one kickback I get is that this is expensive to do and, therefore, we don't spend any time on this. And that is the number one argument - we don't have time for this because this is just one more thing that we have to do. My push back there is that it can be right sized. If you think about this as something that is purely an exercise in, well, I've been told that I have to do it, and therefore I must do it and I'm going to just throw something at it and make it go away, I think that's the wrong approach. You absolutely can put in place a program that is very sound, right? It can be very trimmed down, but you want the right view of what's going on with the models internally. You want to make sure that you know which ones are high risk and important, and you want to know whether or not they work. And that doesn't have to be expensive. It doesn't have to be hiring a consultant. It's just having your eyes on the ball internally to make sure that you know when you when you have trust in the result and when you don't. 

[Zach, 35:17] 

Chase, I think right sizing is probably the right or correct word because it - there's obviously, probably a big difference, I would imagine, with JP Morgan, small risk management versus a billion-dollar bank or $500 million bank. So, I could see MRM being scary. Especially as you work down the asset size food chain. So, is there anything you see in the bigger bank space working its way down or any differences that you're seeing kind of in your travels here that bankers, especially the community banking space, should be aware of? 

[Chase, 35:46] 

You know, I think what we need to be aware of is this trickle down from places where we use models that either we didn't use models before, or we don't necessarily think about them in that way. And a good example would be fintech partnerships. So, I wanted to grow my business, I haven't had an opportunity to originate some new consumer credits and that's what I I decided to do as a management team, but it was difficult for me to do in my local market. I had an opportunity to partner with somebody who could originate these loans for me and tell me that yep, it it'll be monthly pools. You can purchase some, they look like this. Here's some average characteristics of them. And everyone internally high

fived and said yep, this is great because we don't have to worry about originating these on our own, they seem like they'll perform fine and this is a slam dunk. Well, what's going on behind the scenes? This other firm is not just ginning something up from nothing. And every time we're asked to come take a look at what they're doing, they're they're doing some proprietary modeling on their side that is increasingly becoming more complex. They're trying to use ML or machine learning or AI type models to say I can more optimally select customers for you than you can, and then I will extend credit to them and then I will give you that paper and you get to hold on to it and and see how that goes. And if that's not a place where you want to feel like you're trusting the model that selected those customers, then I don't know. We don't even need to be here, right? But that it happens all the time, so do you think about, and do you management team think about, that as model risk? It is, right? That's exactly what it is. Someone has given you a new pool of loans that they've said on average behaves a certain way. And I've seen every year, for multiple times every year for the last three years, models that aren't working that are doing exactly this. So, if your strategy was someone in an ALCO meeting said we need to grow earnings and this is a way you can go about doing that. And everyone said, yeah, this is a slam dunk. Well, you look at the performance of it three years later, it's not been. It's been a loss leader. It's been eating up anything that we thought may be potential earnings on it because it's been underperforming what that model said it should have done when it was originated and for a few of the models we've looked at, it's been with no end in sight. New pools have only done worse than the old cohorts. So, this is a place where early and often, and you talk about where things do and should trickle down from the larger space, it's pushing harder on these partnerships and any place where really the action or the decision-making is is based is model based management has to have transparency into that. That's not necessarily expensive to you so much as it is having the wherewithal to push on that process and say, I don't understand - you have to explain this to me so that I understand what it is that you're doing and why it works. And if you can't convince me that that I understand the way this model is supposed to work and that you can demonstrate that it is, I wouldn't do it, right? That’s that's the point, yeah. I didn't have to pay anybody extra. I didn't have to pay a vendor to come in and tell me that I didn't have to build all the this extra structure and hire all these new people internally and put MRM shirts and hats on them and and that expensive fair. You don't have to do that. But everyone operating at the table and saying alright, this is something that should be top of mind for us and we're going to push on it until we're comfortable, I think that's important. The one thing that you would want to build and then properly have documented that is the governance structure that you want to operate under and that's an an easy thing to do that says because if we want to and and new regulation around this and vendor management is is poignant as well where if we want to have partnerships, if we want to do any of these new strategies with more complex

modeling approaches, here's what we're going to do. And it just sets the ground rules, it doesn't have to be too complex, but you don't want to have a situation that there's no requirement. That anyone have any oversight or understanding of what's going on at that necessary level, right. You've got to be deep enough to know how it works if you don't push hard enough, you never get past the marketing, which is this model works and you should want it and OK, well, that's nice, but I want you to show me more that tells me that this is something that I want to do long term. 

[Zach, 40:39] 

And and if you're not, it sounds like it could be very expensive in the in the long term. 

[Chase, 40:42] 

That's very expensive. These have been some expensive mistakes. 

[Zach, 40:46] 

Anything else on your side? You want to chime in? 

[Vinny, 40:47] 

No, this has been really eye-opening to me and you know, Zach and I are kind of, we're all under the same umbrella of Darling Consulting Group, but we have so little sort of exposure. We do talk as a as a firm internally, we we actually have standing meetings where all of us participate and share kind of ideas collectively, but really really appreciate your time, Chase. I think we're gonna have a bunch of people who are gonna listen to this. There is gonna be some people going what the heck is CECL? And there's gonna be some people are gonna be extraordinarily thrilled to listen to this because I think you really shed light on some really interesting sort of topics today. 

[Zach, 41:20] 

Absolutely. And I think as as with anything, I mean we we want to do a little bit different today just to kind of get the CECL MRM verbiage out there, and I think this is definitely a one on one class and we can do a lot deeper dives as we, you know, next year or or the year after or whatever. But I I I really got a lot of the out of this discussion. I definitely hope our listeners do, too. 

[Chase, 41:39] 

Yeah, me too. This is what we do all day, every day in my group. So it's just wind me up and let me go, and we can talk about it. 

[Vinny, 41:48]

Oh, that's cool. And one thing he did mention is that he said if we locked ourselves in a room, we could have a a compliant model within one day. He'd have to be in the room, Zach. 

[Zach, 42:00] 

I don't think he meant Zach and Vinny. 

[Vinny, 42:03] 

Yeah. Otherwise yeah, exactly. Exactly. Well, thanks so much, Chase. Appreciate it. 

[Chase, 42:05] 

Thanks so much guys, appreciate it. 

[Dana, 42:09] 

On the Balance Sheet is a podcast produced by Darling Consulting Group (DCG). All views and opinions expressed by the hosts and guests are solely their own and may not represent those at DCG. All third parties are independent entities and are not affiliated with DCG. This podcast is intended for informational and educational purposes only and is not considered as advice. All views and its opinions expressed are based on the information available at the time and may have changed based on the current market and other conditions. For more information about DCG, please visit  www.com.darlingconsulting.com or e-mail us at info@darlingconsulting.com. Today's background music is provided by John Sid, the Common Media and can be found on pixabay.com.

The text of this transcript was generated by an artificial intelligence (AI) model, and its organization, grammar, and presentation enhanced by AI, and as such may contain errors or inaccuracies. DCG is not liable for any damages, however caused, that may result from any use of this content.