The Legal Risks of Trusting Artificial Intelligence

What happens when a machine built to predict the next plausible word starts drafting the arguments lawyers put before a judge? 

Gavin Tighe and Stephen Thiele dig into a landmark cautionary tale for the Canadian legal system: the UK High Court's rebuke of international firm Pinsent Masons in Cork & Anor v Smith, where a junior lawyer's 59-page AI chat produced a fabricated insolvency rule that made it all the way into a letter to the court. 

The hosts widen the lens to a Quebec arbitration annulled over hallucinated case law, and to Ontario's own certification requirements for lawyers using AI. 

It's an unvarnished look at professional regulation, supervision, and what "checking your work" still means in Canadian law when the shortcut sounds so convincing.
 
Listen For:

5:12 What actually happened in the Pinsent Masons insolvency case?

9:56 How did the AI chat transcript reveal the junior lawyer's process?

18:19 Where does AI genuinely help versus create legal research risk?

21:38 What does Ontario now require lawyers to certify about AI use?

27:23 Why do law firms need a formal policy governing AI use?

Leave a rating/review for this podcast with one click

Contact Us

Gardiner Roberts website | Gavin email | Stephen email  

Transcript

Copy Transcript Copied

Gavin Tighe (00:00):

I mean, if you put something into AI, what comes back is just like, wow, that's better than anything I could have ever written. It gives you the answer you want. The oldest conman rule in the world is the easiest con to make is the one where people want to believe it's true. 

Hello and welcome to the next episode of Beneath the Law, Gavin Tighe with Stephen Thiele. Stephen, how are you?

Stephen Thiele (00:26):

I'm doing pretty good there, Gavin. I can tell you

Gavin Tighe (00:30):

That this is the real Stephen Thiele, not the robot.

Stephen Thiele (00:34):

Not the AI generated one?

Gavin Tighe (00:37):

That's right. The one, the only Stephen Thiele.

Stephen Thiele (00:41):

Well, I'm doing okay. You also don't look like you're AI generated and I hope you're having a great day, but I understand there are actually programs out there that allow you to create AI attributes for people.

Gavin Tighe (00:57):

Yeah, I could use a few attributes like intelligence. I have the artificial part down, but the intelligence part still alludes me. So today we're going to talk about the advent of AI and its impact on the legal world. It's been pretty dramatic in terms of the number of cases. We talked about it before, but the number of cases that are popping up all over the place like mushrooms after a rainstorm or they leave the sprinkler on too long. AI is the biggest cheat ever. I mean, it's an easy hack for people to sort of not do any work. That's what I think.

Stephen Thiele (01:37):

Gavin, I continue to shake my head at what is going on. And this case in particular, it comes out of England, a very, very well renowned international law firm with a great reputation, like a stellar reputation in terms of its representation of clients. Like shoe,

Gavin Tighe (02:03):

As they might call it in Manhattan.

Stephen Thiele (02:06):

And three lawyers basically walk into a propeller because the junior lawyer on the file is basically having a chat with artificial intelligence to answer a legal question and gets it wrong.

Gavin Tighe (02:24):

Yeah. I mean, okay, so let's just sort of backtrack on that. So what we've got is, I guess we can get into the specifics, but it was an insolvency case, as I understand it. But basically this is not an uncommon situation. You get in a law firm, you get a technical point, you oftentimes will go to the junior lawyers in the firm and say, "I need some research on this or that or the other thing." And they come back with a memo or what have you and they've given you what the law is. And quite frankly, it's not uncommon, particularly in larger firms. It percolates up the food chain. The senior person's name ends up on the brief or on the letter or what have you. They had precious little in terms of the actual creation of that although it sounds good and they go with it.

(03:21):

And in fairness, it would be impossible to review and re-research. Are you sure you got that right? If you did that, I mean, what do you need juniors for? So the whole point of having a law firm and an apparatus in the firm is in order that you have that ability to generate that kind of work because if you have to do all the work yourself, everyone would be a sole practitioner.

Stephen Thiele (03:50):

No, but it's a... Well, Gavin, just on that point, if I think back to my days as an article in student 37 years ago, the senior lawyer did take the time to actually sit down with you and go over your work. And I think what has begun to happen in this evolution of law is that we are working at almost lightning speed on files. Everybody wants an answer the same day or yesterday. And so people aren't doing what would've happened in a very, very traditional practice 40 years ago and people aren't checking. And worse, this case really shows something in terms of standard practice, didn't inform anybody up the chain as to what was happening at the bottom end of the chain.

Gavin Tighe (04:45):

Well, nobody checked.

Stephen Thiele (04:47):

And then nobody checked. Right.

Gavin Tighe (04:48):

So maybe you can give our listeners just sort of a brief background of what happened here. I will say that the particulars of what happened here are interesting, but they're not that different. I mean, you could change the area, change the way it works, but why don't you tell our listeners what went on? And the firm I think was Pincent Masons in London and they certainly took a black eye over what happened here.

Stephen Thiele (05:12):

Yeah, they're headquartered in London. So look, you already alluded to it. It arose out of an insolvency proceeding in England. There was an application that was made, it's called a block transfer application. And essentially the way that I understand that and for those who actually practice an insolvency in England, please don't jump down my throat if I get this wrong.

Gavin Tighe (05:38):

You

Stephen Thiele (05:38):

Should have used

Gavin Tighe (05:39):

AI.

Stephen Thiele (05:39):

Yeah, I should have. But this kind of an application essentially allows a trustee in bankruptcy to have their responsibilities transferred to someone else because they're either retiring or switching law firms. There's a process under the Insolvency Act or insolvency rules in England that permits that to happen. And part of what I guess is standard in the order is that you release that office holder from liability. And so it's clear under the legislation that when there's a performance of an administrative task, the release is not a problem. There's actually an express power under legislation to allow that to happen. But on other issues, i.e. Liquidation issues, that power is not as clear. So the court decided to ask Pincent Mason lawyers, and this is an uncontested application, which I think is important where the authority came from this draft order for the release of both administration and liquidation matters.

(07:06):

And the work went from a, I guess, senior associate down to a junior level associate. The junior associate began to chat with AI. We're not clear what the platform was.

Gavin Tighe (07:22):

It doesn't matter.

Stephen Thiele (07:25):

Quite extensively, there's like a 59 page transcript at the end of the day and answers the question flagging a rule that actually the language in the rule didn't exist, presented it as a quote to the judge in a letter went back up to the senior associate. She didn't know that AI was used, signed off on the letter. She questioned whether it was right or not, but didn't do any other research. The letter goes to the judge and the judge is stunned by what he has seen, goes back to the law firm again to clarify the points and - Well, pointing

Gavin Tighe (08:08):

Out that that's not what the statute said. Right.

Stephen Thiele (08:11):

Well, the judge had done their own research. Well,

Gavin Tighe (08:13):

They looked at it and said that doesn't sound... I mean, that's not unusual. Judges are not... Well, I mean, there have been cases - They're not rubber

Stephen Thiele (08:20):

Stamps.

Gavin Tighe (08:20):

No. Well, there's cases as we've talked about on that too. And there was a recent case in Canada where an arbitrator just bit on a fake citation and that was the grounds for overturning the arbitration.

Stephen Thiele (08:33):

It does happen.

Gavin Tighe (08:34):

Judges do... I mean, for the most part, they go, "Wow, that doesn't sound right moment." And then they go and pull the book themselves and look, that's not what it says. And then they went back to the firm here and said, "Are you sure?" And they came back and said, "Oh, well, we're not - " It was a

Stephen Thiele (08:49):

Summary, but we're sure kind

Gavin Tighe (08:51):

Of. Yeah, but we got it right in substance, but yeah, it's not a verbatim quote. It was completely, as I understand, it was completely just made up. Sounded good though, apparently. But I have to get through twice through the law

Stephen Thiele (09:05):

Firm. But here we are with the use of AI and there's a couple of points. Gavin, my job as you know, is I do background work. I'm a research guy and I've built my career on that in terms of following a very, very traditional research methodology to find answers to legal questions. I know the sources where to find the answers and if I don't, I spend the time trying to learn what sources will help me find the answer. And I read a lot, which is why my eyes are shot now in terms of being able to read without reading glasses or cheaters, but you

Gavin Tighe (09:47):

Get the surgery. I got that. It's

Stephen Thiele (09:49):

Fantastic. Yeah. I don't know. I'm not a big guy on the surgery front.

Gavin Tighe (09:53):

It's a bit creepy when you think about it, but it actually works fantastic. Anyways,

Stephen Thiele (09:56):

We digress. But anyway, here we had this junior lawyer engaging in a 59-page chat with the AI to basically answer the questions. And now in fairness to the AI platform that was used was telling the lawyer to check and verify.

Gavin Tighe (10:15):

Yeah. But I mean, I think this AI is pretty cool. I mean, as the old guy -

Stephen Thiele (10:21):

It's a nice disclaimer of liability.

Gavin Tighe (10:23):

Yeah, I know, but it spins a wicked arm. I mean, if you put something into AI, what comes back is like, wow, that's better than anything I could have ever written. That's fantastic. I'm completely persuaded. And the only other thing I find about if you read it, it tells you what you want to hear. It gives you the answer you want. So it gives you the answer that the you're going to win answer ultimately, which is the oldest conman rule in the world is the easiest con to make is the one where people want to believe it's true. It's like Bernie Madoff didn't make all his money by... Well, he tells people you're going to make money and everybody wants to hear they're going to make money. So it's the same with AI. I find that it will often come back, if you look at it uncritically, it comes back and tells you exactly what you want to hear, which is apparently the same in this case because what the AI came back with was what the answer that the law firm wanted, which was here's the theory to support what you want the court to do.

(11:33):

So that I think you have to be a complete skeptic at this point with AI. It's got the A part right, but the eye is still a little wonky and you need to be the man from Missouri, so to speak, when it comes to AI.

Stephen Thiele (11:49):

As I teach our students here at our firm, Gavin, AI is just an aggregator of information and with, I guess, the added element that it predicts words and that it sounds good.

Gavin Tighe (12:06):

It does sound good.

Stephen Thiele (12:07):

It does sound really good. Right. But just because it sounds good doesn't mean it's

Gavin Tighe (12:10):

Legally correct. Yeah, I think that's really important. And that's

Stephen Thiele (12:13):

The problem.

Gavin Tighe (12:13):

That's such an important point that people think that AI is like this actual thinking being it's not. What it does is it's a predictive model based upon predictability of outcomes and what it does is it's able to do math, math really fast. And what's the likely answer on a preponderance of possibilities as to what would be that? And it does that instantaneously, but it's not thinking so to speak. It's just basically knocking off options and giving you an answer based on what it believes is the predictive model.

Stephen Thiele (12:49):

It's putting a bunch of words together and as a lesson to young lawyers

Gavin Tighe (12:58):

And the other thing too, I always think

Stephen Thiele (13:00):

About is - Stay away from AI. Well,

Gavin Tighe (13:01):

Don't be so quick to run into the propeller. I mean, you're basically when you hand off what you do and the value you bring to the equation to a machine, don't you make yourself completely redundant? I mean, what's the point? If you believe that AI can do a better job than you can, I have to say it does come back pretty good sometimes. You make yourself irrelevant, don't you? I mean, why would you ever want to do that?

Stephen Thiele (13:31):

I don't know about that, but to me, it's almost a reflection of we've lost the ability to socialize and actually talk to one another and bounce off ideas off of real people instead of, "Oh, well, you know what? I can bounce my ideas off of a inanimate AI program."

Gavin Tighe (13:51):

Yeah, no, look,

Stephen Thiele (13:53):

I- It's my best friend.

Gavin Tighe (13:54):

Well, okay, look, I have a different view of it. You and I are of the same vintage. I remember when we first went to law school and it was the first advent of computer searches on legal databases and used to have to go to books in a library, that was it. And then I remember courts were quite adverse to computer searches of databases. Now it's everywhere. Now you're negligent if you don't do computer searches. Now you have way more access to case law than you did when we first started practicing 30 plus years ago, because if it wasn't a reported case, nobody knew about it. Now it's like every single case is online and you can find it. I mean, it is a far superior model and a far broader sampling of judicial thinking in terms of case law and precedent than we started with.

(14:52):

And I think AI is probably in the same infancy that original computer reported cases were.

Stephen Thiele (15:00):

Well, look, one, I hope that's the case in terms of the evolution of AI actually giving correct answers and not giving wrong answers.

Gavin Tighe (15:10):

Look, this boat is believing and if you aren't on... I mean, I think my own view is AI is here and it isn't going anywhere. And I think what we've seen certainly is a lot of errors and stakes and people frankly just being lazy and not reading the cases to check on what AI is doing because... But on the other hand -

Stephen Thiele (15:32):

Or the legislation.

Gavin Tighe (15:33):

Yeah. On the other hand, I think that if as with many other things, there will come a time where if you aren't on the AI train, you're going to get left behind because I know for a fact that many, many large firms are developing their own AI. They have to. And they don't want to be left behind that there is a incredible tool at their disposal that they better use or their competitors will use it and leave them behind.

Stephen Thiele (16:04):

Look, Gavin, I don't disagree with that. Hopefully by the time that that comes, I'm long gone from the practice of law because - No, but it's - Don't count

Gavin Tighe (16:13):

On it with the rate of change if things are going and maybe next week.

Stephen Thiele (16:16):

Well, then I guess I better pack my bag sooner and - Not going anywhere. But I'm somewhat doubtful that we're going to see anything happen next week. But at the end of the day, Gavin, you still have to do the human checks. You can't just be reliant on what is being spit out of the computer at you.

Gavin Tighe (16:38):

No, I mean, I come back to... It's like the same old.You go to court to argue a case and I wish there was a shortcut. I've been looking for a shortcut for 34 years. The shortcuts, there aren't any shortcuts. You got to read the cases, you got to understand it because when you're standing up there and you're the one making the argument, it's not ChatGPT that's making the argument. You're making the argument. When a judge asks you a question, you better be able to pivot and answer it and you have to have some basis of knowledge to do that or else you know what? You look like a fool and you're not effective. So I agree with you. I mean, as much as nobody would like to find a shortcut more than I would, I don't know that there are any. I think at some point in time the information, whether it comes from AI, whether it comes from AR has got to get from the computer screen, the paper, the book, the whatever into your head so that you can actually make the submission.

(17:42):

As long as we're in the world where oral advocacy is how courts are dealt with and as long as we're in a world where lawyers have to stand up and make arguments in front of human judges, I don't know that there is a shortcut, although it is this bright, shiny object at the moment that I think everybody is attractive, including... I would love to see more use of it to narrow stuff down so that perhaps the answer is there's less to read or that there's less, it makes your job a little easier and it saves some clients money in the process.

Stephen Thiele (18:19):

Look, I think there are certain applications of artificial and technology in the law that are extremely helpful in terms of reviewing a contract to find if there's a specific provision in the contract aggregating thousands and thousands of emails and then asking what email mentions Gavin Tai, for example, that is tremendously efficient, but in today's world, to use AI to conduct legal research as a shortcut has significant risk and we're seeing that you talk about judges, live judges, we've got judges who have used AI to produce judgments. I haven't heard that much lately, but - No,

Gavin Tighe (19:14):

You haven't. I mean, in fairness,

Stephen Thiele (19:18):

They're human beds.

Gavin Tighe (19:19):

And I mean, they suffer from the same attraction to the bright shiny object as the rest of us. And I think it's only a matter of time before you're going to have cases in this worldwide phenomenon in terms of lawyers being caught wrongfully using AI. But it's only a matter of time before a judgment is found to have been written by an AI bot. And I mean, you can hate the sin but love the sinner as the saying goes and you can understand why people would do that. It's attractive because it is easy and it's slothful, but it's an understandable sin. People look for shortcuts all the time and judges are people.

Stephen Thiele (20:04):

I know, I guess maybe I'm an outlier because I always look how to do things the hard way instead of the easy

Gavin Tighe (20:11):

Way. No, but that's why you're good at what you do.

Stephen Thiele (20:12):

Well, thanks very much. I appreciate that.

Gavin Tighe (20:16):

But that dose of skepticism I think is what's missing. I mean, we also have a generation that is raised on if it's on Google, it must be true. And I think that that is one of the things that we... I personally have a lot of difficulty believing a lot of the things you read online and most of it's baloney anyways. Most of it's not true, certainly most of the things you read on social media. I wouldn't believe 10% of it.

Stephen Thiele (20:44):

No, I think it's healthy to have a dose of skepticism, right? With respect to anything.

Gavin Tighe (20:51):

Sure. And I mean, I think particularly with regards to law, when you're making the argument, I think ultimately one of the things that lawyers, especially young lawyers, have to understand is your name is on the brief or you're the one standing up in court making the argument. It's

Stephen Thiele (21:09):

Your

Gavin Tighe (21:09):

Argument. You know what? If you used AI or whatever to get it, whatever, but you own it at the end of the day and if it's wrong, it's going to land on your head. And if it's wrong because you didn't do the work to read the cases that you put your name to, too bad for you're warned. I mean, now for example, in Ontario, you've got to sign the fact that says you've checked the sources. You've got

Stephen Thiele (21:38):

To give - But that doesn't happen all the time, right?

Gavin Tighe (21:40):

People don't do it.

Stephen Thiele (21:41):

Right. People don't do that.

Gavin Tighe (21:42):

They're required to do it.

Stephen Thiele (21:43):

They're required to do it, but they don't.

Gavin Tighe (21:45):

But I've seen it on these cases where I've been dealing with a number of these recently where lawyers have been hauled up on the carpet by the court and one of the questions is, why didn't you sign the certificate? We know now that these were hallucinated authority, you didn't sign a certificate, coincidence. I mean, you didn't do it and you didn't say you did it when you were required to. It really is a road to perdition as far as how to quickly lose your law license.

Stephen Thiele (22:17):

Well, you know what? It's very interesting. I haven't seen a case yet where a judge says that the certificate wasn't signed. I don't know if clerks at the court now before they allow documents to be uploaded to the caseline systems or filed with court of appeal are rejecting it if you don't have the certificate signed. I haven't heard of that yet, but maybe that has to happen.

Gavin Tighe (22:42):

Well, I've seen it happen. I've had a case where they wasn't signed and then they turned out that they were hallucinated cases, reported case.

Stephen Thiele (22:49):

Right. But the material wasn't sent back to the lawyer that fed the material to say, "Hey, sign the certificate." No,

Gavin Tighe (22:55):

Because it went to the court. That's

Stephen Thiele (22:56):

What the

Gavin Tighe (22:56):

Judg said. There we go. "These aren't cases aren't real and you didn't sign the certificate."

Stephen Thiele (23:03):

So we're seeing that.

Gavin Tighe (23:04):

We are seeing that. We have seen that. I mean, certainly certain judges, certainly in Ontario have been pounding the drum and speaking with them, they can't believe it's still happening, but it's going to keep happening. I mean, I have to say, I mean, I think it will continue to happen because I think that as soon as there is an easy route and as I said, I think it's a sin of laziness and people will take the path of least resistance. If it's an easy way to do it, they're going to do it.

Stephen Thiele (23:35):

Well, it's either a symptom of laziness or a symptom of working in a high pressured environment and having to turn around and answer in a short period of time. Yeah. There's a

Gavin Tighe (23:46):

Ton of pressure, right? Tons of pressure. I don't think we're excusing the behavior when we talk about this at all. I mean, certainly if anyone's taking that, that's what I'm saying, that's not what I'm saying, but I cannot excuse the behavior but still understand it. And I think that there's an enormous amount of pressure. I mean, this was as we called a white shoe law firm out of the UK. It would be interesting to see their accounts to see if the lawyer billed for preparing the memo that was wrong and if they refunded the client for the fee they charged, but you can see where lawyers who have an enormous amount of time pressure and monetary pressure to churn out work where they may or may not be getting paid for it and they have concerns about how much they're going to be paid all sorts of family law where there's all sorts of pressure, there's not a lot of money to go around and there's an enormous amount of work to be done for what is relatively small amounts of money.

(24:47):

I mean, that pressure, that economic pressure on younger lawyers in particular, you can see why this becomes very, very attractive. They're overworked. They're worried that they're going to get paid and this just seems like an easy way to get the job done.

Stephen Thiele (25:05):

Well, and especially too, Gavin, I don't know the UK experience going to law school, but certainly we turn out lawyers here in Canada. Some of these kids are coming out of law school, hundred thousands of dollars in debt, right? Yeah. So there's lots of pressures immediately in their first year of practice. And

Gavin Tighe (25:29):

That

Stephen Thiele (25:29):

Doesn't - 1600, 1700 hours of billable work, multiply that, add three hours for every 10 hours of work, right?

Gavin Tighe (25:37):

Yeah. And that's in a large firm model where the billable hour

Stephen Thiele (25:40):

Matters.

Gavin Tighe (25:42):

In the trenches of your average lawyer, I mean, most lawyers don't work in large firms, but the vast majority are sort of hand to mouth. They get paid oftentimes a percentage of collection. And when you're talking about collected money, I mean, you're really sitting there going, "I really want to spend 10 hours reading all these cases, which the client will maybe pay me a couple hundred bucks for because that's all they got, or am I going to run it through an AI model and get the thing out the door?" You can understand that the pressure with respect to dollars and fees collected is enormous, even more so than it is in just doing... If your pressure is just to put the time in, then you've got an incentive to put the time in. But if your pressure is to actually collect money, I mean, that's a far, far more difficult thing to do than just bill hours.

Stephen Thiele (26:35):

Yeah, no kidding. It's all part of the business of law.

Gavin Tighe (26:39):

It is an enormous... And part of the business of law, but what we're seeing is... And I'm hopeful that... And I think that AI will evolve. I think it's going to be around for a long time. I think it's going to replace a lot of lawyers as it becomes more dependable, just like computer modeled research did. But you still got to read the cases and you still got to understand.

Stephen Thiele (27:07):

It's not going to happen next week.

Gavin Tighe (27:09):

Well, maybe not next week, but the pace of change is to me -

Stephen Thiele (27:13):

It is pretty

Gavin Tighe (27:13):

Quick. Man, it reminds me of one of those old bugs money cartoons with the snowball rolling down the hill. It's just getting bigger and bigger and faster and faster.

Stephen Thiele (27:23):

So we're coming up here on probably our time to end this discussion, but one of the things that I think is really important for law firms is that they have to have... And accounting firms too and other professional firms, you have to have robust AI policies in terms of when should somebody use it if somebody's using it to report that up the chain because that is really important. And I think that's one of the biggest failures here is that nobody was actually paying attention and asking the questions.

Gavin Tighe (28:00):

They're passing it off as their own work. Well, I tell you what I think with the message here is if you're going to pass AI off as your own work, be careful of what you wish for because if it's wrong, you wear it. And ultimately, I think if people get that message, then maybe we'll have a little bit more surveillance and a little bit more healthy skepticism as to the bright bubbly object and the wonderfully prosaically written memos that come out of AI. Anyways, another salutory tale and cautionary tale of why if no one is above the law, everyone is beneath it. Thanks, Stephen, for a great discussion. Thanks to our producer, Dougs Downs. And also, please rate us. We love to know that somebody's listening as we bellow in the wilderness about these erudite topics of legal interest, at least to us. And if they're of interest to you, let us know too.

(29:00):

And if they're not, tell us that too. Maybe we can actually talk about something you are interested in and entertain and inform, which is our objective. Anyways, until next time, thanks for listening.