AI Legal Advice Could Cost You Your License

ChatGPT can write a convincing legal argument in seconds, and that’s exactly why it can be so dangerous when your professional license is on the line. With new technology, new patterns are emerging. People are treating AI like their lawyer, only to get blindsided by missed deadlines, incorrect filings, and “authority” that turns out to be completely made up.

We talk with Associate Attorney Amy Cadwell and Legal Support Manager Melissa Hooper of Bertolino Law Firm about what is driving this trend and how quickly it can go wrong. From hallucinated case citations that have resulted in attorney sanctions to clients pulling up chatbots during attorney-client conferences, we unpack the real risks of using AI for legal advice, licensing board responses, and administrative law strategy. We also explain why licensing board cases are so nuanced: each agency has its own processes, constantly changing rules, and procedural traps that a generic chatbot just cannot track reliably.

You’ll leave with a clear playbook for using AI responsibly in a licensing board investigation or complaint. We share safer, practical uses like building a chronological timeline, generating key questions to ask your lawyer, and translating confusing documents into plain English. And if you've already drafted something with AI? We cover the next steps that matters most: stop, get it reviewed by a qualified attorney, and be fully transparent so your legal team can protect you.

Stay up to date in your professional industry! Subscribe to Know Your Regulator, share this episode with a colleague, and leave a review with the topic you'd like to see us tackle next.

Get more information, details and resources on Know Your Regulator - https://www.belolaw.com/know-your-regulator

Transcript

Purpose And Legal Disclaimer

SPEAKER_00 0:00

The purpose of this podcast is to educate and inform. It does not provide legal advice. It does not create an attorney-client relationship. While the host is not a lawyer, the content is overseen by licensed counsel. If you need help with a legal matter, you should always consult with a qualified attorney.

Why People Trust ChatGPT

SPEAKER_03 0:15

We've all done it. Ask ChatGPT or another LLM something random, maybe something like plan my vacation, write an email, explain this concept to me like I'm five, but lately we have been seeing it being used for a different purpose. People using AI as their legal strategy. And so today we're going to talk about how and why that can go very, very wrong. This is Know Your Regulator. I am your host, Simone Murfrey. And to help me break down this trendy problem is Bertolino Law Firm's Associate Attorney Amy Cadwell and legal support manager slash paralegal extraordinaire Melissa Hooper. They both have firsthand experience unraveling some of the issues that AI can create in these modern age cases. Amy, MJ, thank you guys for joining me. Of course, thank you for having us. Absolutely. Well, let's kind of set the stage for our audience. Amy, we'll start with you first. Why are people turning to tools like ChatGPT for legal help?

SPEAKER_01 1:19

It's fast, convenient. Um, it gives those structured answers. And people, you know, they just say it sounds legal. It and it's got to be right. I mean, it sounds really, really good. I've used it before because it sounds so good.

SPEAKER_03 1:32

It definitely speaks with authority and with that confidence, right? That makes you feel like whatever answer it's giving you is correct and it's factual and it makes sense, and it'll kind of give you like clues, even like extra sentences to sort of to sort of boost the answer that it gives you so that it seems more legitimate, you know?

SPEAKER_01 1:53

Right. It's got that legalese and the lingo down pat.

Dangerous Prompts With Real Stakes

SPEAKER_03 1:57

Absolutely, yeah. No kidding. We've joked before um with all of us about some of the things that you know people ask AI to do. And MJ, you actually had some really good uses that you had mentioned, and some that we kind of then cringed at too. But what are AI or what are people asking AI to create? What are they tasking it to do?

SPEAKER_02 2:22

Some very easy ones, or some of my favorite ones are like help help me create a recipe or a meal plan for tonight. But some of the other very um egregious or problematic ones are going to be asking it to diagnose any medical issues. It's not gonna know how to do that. Or how do you recover from a recent surgery that you've had? That's crazy. Everybody's different. You don't know. And then, in addition to that, especially where it affects anything dealing with law, is writing your own legal defenses and generating legal arguments. Those are definitely for the attorneys, they are the skilled professionals for those.

AI Citations That Never Existed

SPEAKER_03 3:04

Absolutely. And I think there's been there was some kind of trend, trendy post or viral post that went around where people had put giant googly eyes on their eyes, and then they asked Chat GPT to kind of like ask them, like, what's wrong with my eyes? What's going on? Can you help me? And it took Chat GPT a really long time to come up with or to figure out that they were just googly eyes that they had taped on their eyes. So um, yeah, to your point, you really can't like trust what ChatGPT says. And then when we're talking about mixing in your professional license in with it, it stops being so funny because it can go wrong so very, very quickly. Amy, we know that really the bigger issue that's happening in the legal world right now is we're hearing about attorneys getting in trouble for AI generated filings. Can you talk to us a little bit about what is happening there? What's going on? Why are they getting in trouble for something so silly sounding?

SPEAKER_01 4:04

Oh, you know, Simone, it it may sound silly, but there was a case, I believe it was in New York, and an attorney had used AI, and so he walked into court, he presented his arguments, but then the judge went looking for the cases he was citing, and those cases were made up. And that that attorney, I don't think he got disbarred, but he got in big, huge trouble for that. If the attorneys are getting into trouble for that, you can imagine, you know, Joe Blow off the street, here he is with his professional license. Nobody is gonna be okay with you making up a case if you walk in saying waving your Chad GPT case that that's been made up that to support your defense. That's just not gonna cut it. I mean, wasn't he fined? Yeah, oh yeah, he was fined, but then they also did some other things, and I don't remember exactly what they did. Um, I know he was disciplined by the state bar.

SPEAKER_03 5:08

And then it kind of comes down to like, do you really want to hire a lawyer too who's gonna use ChatGPT as their argument, you know? Um I'm right.

SPEAKER_01 5:19

Oh you bought an attorney who went to law school and not Chat GPT. Exactly. I mean, but you know what? I mean, the truth of the matter is ChatGPT is great for some really um for some novel ideas out there. I use Chat GPT. If I'm writing something and I I need a paragraph to sound a little more polished, I'll use uh Copilot or Chat GPT, I'll use AI because it's a handy tool to have. But what we need to remember is it's a tool. It can't take the place of your lawyer or your doctor. It should never take the place of those because it's not a trained lawyer. It doesn't know, okay, this statute applies here, okay, this case applies here. Oh, you know, we need to make this argument here. We need to make this very important ultra virus argument here, which if you come talk to us, you'll hear ultra virus uh bandied about. And I bet you won't find that on Chat GPT.

Chatbots Showing Up In Hearings

SPEAKER_03 6:17

Exactly. MJ, what are you seeing? I know that you have got to be, you know, seeing something else and and all of this AI use.

SPEAKER_02 6:29

Well, we definitely see a lot more um of our clients consulting um chat GPT andor co-pilot, even during our attorney client conferences. And we strongly advise not to do that because again, as we just got done discussing, chat GPT is not going to know the specifics and the very fine critical details that your trained professional attorney is going to know. So we do always like to try to remind them you've hired a professional attorney who's gone through schooling and lots of continuing education, not ChatGPT. Are they caught up with all the the recent rules and changes like we, our office follows on a weekly basis? Um, so um that was a big point. But I also wanted real quickly to go back onto something um quickly, if I may, Miss Moan. Because of that one court case that Miss Amy had referred to a few minutes ago. Anytime people are being scheduled for hearings now, in the orders from the judge, they are specifically putting wording in those stating to please make sure to it you are not using ChatGPT in your filings. So it is being warned everywhere not to do it.

The Cleanup After AI Missteps

SPEAKER_03 7:50

And it's kind of like one of those things where you get like a um gel silica pack and you're like, do not eat. Why would who's out here eating it? It's like, no, there are people out there, and sometimes you do have to very blatantly spell it out. Hey, watch yourself, don't, you know, big step below you, don't trip as you're walking out the door. Um, and and uh at this point, lawyers should know better. Amy, you have seen what it looks like when someone comes to us after trying to handle things themselves using Chat GPT or some sort of large language model. What does it look like when you're kind of managing that type of case? Someone comes to you and they're already they ventured into this process and they're essentially using Chat GPT or AI as their lawyer, right?

SPEAKER_01 8:47

I, you know, someone, they come and they hand us, it's like they bring us this big load of documents that are in this jumbled pile that's a mess, and they want us to clean it up for them. They've been sub submitting these responses to the board that are the wrong documents in the wrong format with the wrong argument. They've made admissions that we really wish they hadn't done. They've missed deadlines, they have missed out. I mean, for instance, we had just had a case. Thank God we we we were jumped on the case in time to be able to file the motion in time. Had we been one or two days later, we would have missed our opportunity because under the there's certain rules you have to file within a certain amount of time period, and and there's so many different deadlines, there's so many nuances and intricacies. It's not just there's one law and you have to follow it, it's rules and statutes and legislation and I mean and case law. It's it's a whole gamut of of everything that we are having to put together, and it's you have to put it all together. These are map we're map makers. We're not we're not just you know, we're not just out there for fun throwing out arguments with each other. Right. So um, you know, when we when you bring us your little pile of of little your little your little shred of map, uh the we'll we'll take it all apart and we'll put it back together and see if we can make it pretty again for the regulator and uh see what we can do to put it back together and get you on your way back into your work so that you can and keep going.

Safe Ways To Use AI

SPEAKER_03 10:30

But that could be costly, I would imagine. I mean, that's just more more build time and maybe had um like you had mentioned, MJ. You know, you hire a lawyer, you trust them to do this job for you. That's what you do. You trust them to do the job for you, and you don't try and um do it alongside them with something or someone who didn't go to law school. So absolutely. And MJ, you and I have talked a lot about the great uses of AI. We know that it's a great valuable tool, and there can be so many things that it can be helpful with. Where can it actually be helpful when it comes to a legal case? What are the things that um individuals or licensed professionals can use AI for? That doesn't necessarily mean that they're kind of working alongside their lawyer or even undermining, you know, the expertise that their lawyer has.

SPEAKER_02 11:35

So, some some basic good uses. A lot of the times with our clients, all of our attorneys do ask for a written chronological timeline. So with ChatGPT and or Copilot, that would be fantastic because you could type in what you want. And even if your thoughts are coming out um unorganized or you've jumped a month ahead, it's gonna be able to wrangle it back in and put it in that narrative in that in chronological order. So that is helpful. If you are unsure of what type of questions to be asking your attorney, you can tell ChatGPT andor co-pilot, hey, I'm dealing with Board of Nursing, or I'm dealing with Texas Real Estate of Commission, and I have this complaint, and this is what they're stating. Can you help me come up with questions to ask my attorney to obtain our next steps or where we're at in the process? Um, another positive or or good use is gonna be if you have a document that you truly don't understand and seems very Greek to you, you could ask it to summarize it and make in a way break it down and talk to you like you're a five-year-old. And then you'll be able maybe to understand the concept and be able to express your concerns and again attack the roadmap the attorney is providing for you.

SPEAKER_03 13:03

Those are all really, really great uses. Are there any that you would say do not use AI for this other than the obvious of like using it to um to be your lawyer and kind of draft and and file or you know, give you a deadline, make these sort of legal analysis for you?

SPEAKER_02 13:27

So not to beat a you know, beat a dead horse again, but again, legal analysis. Chat GPT is not up to date with all the current proposed, adopted, withdrawn rules. Each of the agency has this stuff is moving quickly and changes are made quickly. So you really can't guarantee Chat GPT is staying current with those, no matter how realistic it seems. Another one seems very obvious, but a lot of people seem to make this mistake. Writing responses to your regulatory board. Are you proficient in each of the agencies we deal with? I seriously don't think so. We do. We follow and track these agencies on a daily basis. And then the last one, and I'll leave y'all alone, is interpreting the codes, the statutes, and the rules. Again, this is why you've hired a professional who has gone through the schooling and the training and attends all those boring CE classes, but that is why you've hired these attorneys.

What To Do After Using AI

SPEAKER_03 14:29

Yeah, no, it's it's very nuanced. We know this, but that's why we're trying to convey to you, audience, that this is a very nuanced process. It it is it comes down to your individual situation and your individual board. Each board has a different process. Um, they the way that they're structured could be different, the way that they're governed could be different. I mean, there are so many things that happen behind the scenes when it comes to just the overall schematics of of the regulatory world. So um to that, it's it's something where you are a forever student. You're always going to be learning, things are always going to be changing, and you've got to stay on top of that. So, Amy, let's say that someone has already used AI to draft something or respond to the board. What should they do in this situation?

SPEAKER_01 15:25

Well, first, they need to stop and don't go any further because if you do, you're making a big mistake. You need real legal review. Take that to an attorney, let them look it over for you because you uh cannot rely on Chad GPT. You need a real lawyer looking at it because it needs actual human eyes on it. Yeah, absolutely.

SPEAKER_02 15:50

Simone, can I also Oh, go ahead.

SPEAKER_01 15:52

I was just gonna say be transparent with your attorney. Don't hide things from them. You there's the attorney-client privilege, you're in a safe place. Don't lie. That's dumb. Don't lie. Be transparent. Super important.

SPEAKER_03 16:09

Is that what you were gonna say, MJ?

SPEAKER_02 16:12

Yes, ma'am. I we see it all the time where the clients are afraid of looking stupid or they're scared. But when you aren't transparent with the attorney, it could come back and bite you in the middle of the thing, and we won't be able to backpedal. So it's we always stress, please be open and honest. Like Miss Amy says, it is a safe place. There is attorney client confidentiality, but you can't withhold things because it could very possibly cause a negative impact on your case.

SPEAKER_03 16:45

Yeah, you can't defend what you don't know, right? Exactly. Yeah.

SPEAKER_01 16:51

And I So well, AI, go ahead, Amy. I have I have had more than one time where I have had a client on the witness stand and they have been asked a question, and they answered not the way I had they had answered me in the past. And I had told I was sit would sit there and I would think, if only you had told me that, I could have I could have prepared you. But now you're in the hot seat, and I don't know how to answer that, I could have told you how to answer it. So be transparent. Yes. Yeah. It's important. So important.

Final Warning And Sign-Off

SPEAKER_03 17:27

Yes. While AI can be incredibly useful when your professional license is on the line, it does not replace real legal strategy, legal experience, or real legal judgment from an attorney who went to law school. If you're facing a complaint or an investigation from a licensing board, make sure that your last line of defense is a real attorney, not an AI chatbot. MJ, Amy, thank you both so much for joining me this afternoon and breaking down where AI can be useful and where it can bring you a load of trouble. Thank you to our audience for tuning in to Know Your Regulator. And until next week, stay inspired and continue engaging with your regulatory agency. Thank you, ladies. Thank you.

SPEAKER_00 18:17

Know your regulator, the podcast that inspires you to engage.