Cross-examining AI: Use of AI by public authorities and key IP issues

This is the third episode of "Cross-examining AI", HSF Kramer's podcast series on disputes and AI where we unpack the key developments that are shaping litigation. In this episode we look at the use of AI by public bodies following a recent judgment on the lawfulness of the Metropolitan Police’s live facial recognition technology, as well as discussing some of the key IP issues that AI developers and users are currently confronting. This episode is hosted by Martin Hevey, a senior associate in our disputes team, who is joined by Andrew Lidbetter and Jasveer Randhawa from our public law and regulatory team, and Peter Dalton, a partner in the cyber security and IP practices.

Below you can find links to our blog posts on the developments and cases covered in this podcast.

• Civil Justice Council publishes update on consultation responses regarding use of AI for preparing court documents
• Lessons for AI use by public authorities in Divisional Court’s decision on lawfulness of facial recognition technology

Transcript

Welcome And What’s Ahead

SPEAKER_03 0:09

Welcome to the third episode of Cross Examining AI, HSF Career's podcast on disputes and AI, where we unpack the key developments of shape and litigation. I'm Martin Heavy, a senior associate in our dispute resolution practice with a specialism in technology disputes. And today I'm joined by Andrew Lidbetter and Jazz Randawa from our public law and regulatory team. And Pete Dalton, a partner in the cybersecurity and IP practices. Jazz and Andrew are going to be looking at the use of AI by public bodies following a recent judgment on the lawfulness of Metropolitan Police's live facial recognition technology. And Pete's taking us through the key IP issues vexing AI developers and users alike. Welcome to

AI Hallucinations In Legal Drafting

SPEAKER_03 0:51

you all. Before we get started, I wanted to briefly mention a couple of recent points of interest. Firstly, a recent case involving a city law firm highlights the dangers of AI use in legal work without proper supervision. A junior associate used AI to draft two letters to the court in an insolvency matter, citing a provision of the insolvency rules that did not exist. He was hallucinated by the AI tool. The junior failed to verify the output against authoritative sources, and the supervisor didn't ask whether AI had been used. The consequences had been serious, not least a lengthy public judgment, naming some of the lawyers involved, self-referrals to the SRA, and consideration by the court of contempt proceedings. So just a reminder that AI can sound convincing while being entirely wrong. Every citation must be checked. Juniors using AI tools need proper supervision and always consider whether AI is the right tool for the task or which AI tool might be best. Don't overlook traditional legal research platforms.

Civil Justice Council AI Update

SPEAKER_03 1:54

Secondly, earlier this year, the Civil Justice Council published an interim report and consultation on whether rules were needed to govern the use of AI by legal representatives in preparing court documents. The consultation generated significant interest, and the CJC has now published an update on its findings. The key takeaway is that there is strong consensus among respondents that for professional legal drafting, that is, pleadings, skeleton arguments, and similar documents. No additional AI-specific requirements are necessary on the basis that existing professional responsibility frameworks are sufficient. The principal area of ongoing consideration is the use of AI in preparing witness statements, where views differ on whether or what, additional safeguards or disclosure requirements are needed. The update also flags expert evidence and litigants in person as raising distinct issues requiring further attention. Herbert Smith Freehill's Kramer contributed to the consultation, so do get in touch if you'd like to discuss the update. We'll include a link to the CJC's update in the episode notes.

What Live Facial Recognition Does

SPEAKER_03 3:00

Now let's turn to the use of live facial recognition, also known as LFR, and a recent judgment on the lawfulness of the Metropolitan Police's use of it. Jazz, can you set a scene for us? What's this case about and why has it attracted attention?

SPEAKER_00 3:14

Yeah, of course. So the technology in question essentially uses cameras to capture images of people and then uses AI to analyse those images and compare them with the details of people on watch lists, so basically people of interest to the police. The case is called Thompson and Carlo against the Commissioner of the Metropolitan Police. And Mr. Thompson had been stopped following a mistaken LFR match with someone who was on a watch list. It's also worth saying that the Equality and Human Rights Commission intervened in the case, really emphasising the significant risks arising from rapid AI development if it's left unconstrained by domestic and international law. So why is it important? Well, there's been very few cases so far looking at public body use of any type of AI. So this is a really significant judgment coming at a time when the promise of AI is matched really only by the scale of the concern it generates, and when we know that there's a real push for AI to be used by public bodies for the sake of efficiency. The government, for example, has recently unveiled an AI tool to speed up planning applications, including actually making the initial assessment. And we know that many commercial and economic regulators are already using AI for their own functions. So in this case, the divisional court was called upon to deliver a judgment effectively giving guidance on the frameworks and safeguards governing the use of LFR, but the implications extend beyond that particular technology, potentially reaching other forms of AI used by public authorities. And the judgment also lands right in the middle of broader conversations about how human rights can be protected in the age of AI. So, for example, the Joint Committee on Human Rights is examining at the moment whether the UK's existing legal frameworks are truly equipped to handle AI.

SPEAKER_03 5:00

Thank you, Jazz.

Human Rights Claims And Bridges

SPEAKER_03 5:01

Andrew, what were the arguments the claimants used to challenge the legality of the policy?

SPEAKER_01 5:06

The claimants argue that the LFR policy violated their right to private life under Article 8 of the European Convention on Human Rights, and that's part of UK law through the Human Rights Act. And they said this because they said it wasn't in accordance with the law. They also raised Articles ten and eleven, which relate to freedom of expression and assembly, for essentially the same reason. The focus of the case was on the lawfulness of the policy rather than other aspects.

SPEAKER_03 5:38

And this case did not emerge in a vacuum. I think there was a key precedent.

SPEAKER_00 5:43

Yeah, it was a case called Bridges, decided by the Court of Appeal in 2020. And that case held that South Wales police's use of the same technology wasn't in accordance with the law, because it left too broad a discretion to individual officers. So, for example, it didn't specify who could be placed on watch list or the criteria for where to deploy the technology. And that meant that decisions were effectively dependent on the will of those applying the power rather than being foreseeable.

SPEAKER_03 6:12

So how did this case compare with bridges?

SPEAKER_01 6:15

Well the court relied heavily on bridges but reached the opposite conclusion when applying the law to the facts. It confirmed that any unrestrained exercise of power is not in accordance with the law, but found that broad discretion isn't automatically unlawful as long as there are appropriate safeguards, which can be set out in law, policy, or a combination of both. The court also reaffirmed an important point from Bridges that the more intrusive the measure, the more precise the authorizing legal framework has to be. Jazz, you mentioned foreseeability. How does that fit in?

SPEAKER_00 6:54

Well, for something to be lawful, it generally needs to be accessible, foreseeable, and compatible with the rule of law. The key question here was whether the policy left such broad discretion to individual officers that in practice it was dependent on their will rather than on the law itself. If a measure fails the foreseeability test, then it just isn't lawful. You don't even get into issues of whether any interference with human rights was justified and proportionate. That only comes later if this initial gateway is passed. And here, interestingly, the claimants didn't even challenge proportionality. They just focused on lawfulness.

SPEAKER_03 7:31

And Andrew, how did the policy clear that bar and what made it different from bridges?

Foreseeability And Embedded Proportionality

SPEAKER_01 7:35

Well, Martin, unlike in bridges, the policy directly addressed the questions of who, why and where. The conditions, objectives, and locations of deployment of the technology. Critically, it expressly required consideration of proportionality and convention rights. This wasn't just a statement of background principle. Proportionality had to be actively assessed at the level of each individual deployment, including whether less intrusive measures could achieve the same objectives. So in this case, the court was satisfied that the policy provided sufficiently precise and practical guidance rather than just paying lip service to human rights.

SPEAKER_03 8:23

What did the court say about the future trajectory of AI and LFR?

SPEAKER_00 8:27

Well, the court acknowledged that LFR uses and the underlying technology will inevitably evolve, but it also made it clear that it was only concerned with the specific policy under challenge as it then stood. It didn't want to start speculating on future applications. And that approach leaves important questions open about how courts will handle other types of AI systems in future challenges, which is really precisely the kind of gap that the Joint Committee's inquiry is looking to address. And similarly, we know that the Law Commission is also looking into public sector use of automated decision making.

SPEAKER_03 9:01

What practical lessons does this judgment offer public authorities deploying AI?

SPEAKER_01 9:05

Several points. Firstly, the judgment confirms that using new technology doesn't affect the legal requirements. Interferences with convention rights must be in accordance with the law regardless of whether the technology is at a trial stage or not. Secondly, broad discretion over the use of AI is likely to be allowed where there is a clear framework addressing who is affected, why the use is justified, and details such as where or when it may occur. Thirdly, proportionality must be embedded operationally, not simply referred to in policy documents. It won't be enough for public authorities to just mention convention rights. The court's going to examine whether the policy gives practical guidance that is capable of constraining the discretion of individual decision makers.

SPEAKER_03 10:04

And Jazz, a final word on the broader governance implications.

SPEAKER_00 10:08

Well, although, as we've said, the court confined itself to the specific policy before it, I think this case does have obvious significance for AI governance and use by public bodies generally. The framework that it articulates clarity of scope, constrained discretion, and embedded proportionality should really inform how public authorities approach AI deployment across the board. They need to not only think about, but also be able to evidence

What This Means For AI Governance

SPEAKER_00 10:35

and demonstrate, if challenged, that they understand and have weighed up the risks and implications of using AI. They've taken steps to minimise or mitigate those risks, or at least acknowledge them and justify why they think those risks are necessary in the public interest. The Joint Committee's inquiry touches on some of the really important issues that public bodies need to have in mind, like the possibility of bias, explainability, and accountability of decision making. And ultimately, I think the combination of the Joint Committee, the Law Commission, and case law that will slowly build up in ad hoc cases like this will all help to shape the context around public body use of AI.

SPEAKER_03 11:14

Thank you both. That was really interesting. It sounds like the case will be an important reference point for public law practitioners and AI governance specialists.

Three Big AI And IP Flashpoints

SPEAKER_03 11:22

Now, moving to the world of IP. Pete, where are we seeing the key intersections between AI and IP?

SPEAKER_04 11:29

So there are three key areas in which we're seeing a lot of activity and discussion at the moment. One, the inputs into IP systems and whether AI owners are infringing on the rights of IP holders. Two, the risks AI users are taking on in respect of IP infringements arising from their use of AI systems. And three, whether IP rights can be claimed over the outputs generated by AI systems.

SPEAKER_02 11:58

Let's start on the first and go from there.

Training Data Scraping And Copyright Suits

SPEAKER_04 12:01

Sure. One of the largest areas of legal dispute to date relates to the development of these large language models, specifically in the need to ingest huge quantities of data in order to train the systems. Training is often measured in tokens of data, and to put it into context, the latest frontier models are trained on approximately fifteen trillion tokens. That roughly equates to eleven trillion words, which is approximately two thousand five hundred times the entire English Wikipedia, all listening to continuous human speech for somewhere over a hundred and fifty thousand years. To achieve this, content has been scraped from the web in vast quantities. Huge volumes of books and academic works have been fed into the systems en masse, and in the case of systems capable of image or audio generation, vast quantities of images and recordings. Much of this data has been taken and used without seeking the consent of the content owners. This has led to a huge amount of concern amongst IP owners, and cases have been launched in the UK, US and globally alleging copyright, infringement and trademark infringement, as well as other forms of infringement. In the UK, Getty Images sued Stability AI for alleged infringement of its vast library of images in a case that's currently on appeal to the Court of Appeal, with the High Court having awarded victories on both sides. In Germany, GEMA, Germany's music rights organization, has achieved victory against open AI in respect of claims of infringement of copyrights in music lyrics, a case currently on appeal, and it has separately sued an AI music generator for infringement of copyright in its music library. In the US, there's no spate of litigation, some in the form of class actions, some brought by a diverse range of media organizations and other content owners, including the New York Times, other associated newsgroups, and music labels including Warner, Sony and Universal, and authors and literary groups. Many of these cases are still pending, while others have settled or resulted in licensing deals. Anthropic, for example, recently paid assessments of $1.5 billion in respect of claims that pirated works have been used in training data sets. So how do these cases tend to play out? Well, I think the fundamental issue in all of these cases is that AI companies maintain that they need unfettered access to vast quantities of data, and that it's unworkable to have to seek permission from copyright owners en masse, including by way of opt-out schemes, or indeed for them to evaluate whether the data they're taking is legally made available, for example, if it is pirated.

User Liability For Infringing Outputs

SPEAKER_04 14:40

Legal defenses they have raised to date tend to rely on fair use doctrines in the US, or arguments as to territoriality, or nuances of irrelevant infringement arguments. Different jurisdictions have taken different approaches, and I think it's fair to say that in the US and the UK the legal position still remains unclear. In the UK, successive governments have consulted on plans to create copyright exceptions to allow AI training, only to back off in the face of pressure from content owners. In the EU there is a text and data mining exception, which allows the use of copyright material for training in some contexts, but it's not a complete defense. And other jurisdictions have gone in the other direction. Those such as Singapore have sought to court AI developers by league legislating to legalize the use of protected works and training. Ultimately, though, there remains significant uncertainty in many jurisdictions as to whether the training of these systems fundamentally involves IP infringement, and the global situation is still highly fragmented. Sounds very messy.

SPEAKER_03 15:41

How is this playing out in terms of the liability of users?

SPEAKER_04 15:44

Well, that is another unknown. A big risk for users is if an AI model generates an output, which itself infringes the IP of a third party. We've seen several of the cases against AI owners also allege infringements in the outputs. Many of these have not reached trials of yet. In cases that have, there have been some seemingly contradictory results. Ultimately, though, in many cases, users are contractually at risk because AI platforms do not accept any liability for the outputs generated by user interactions with their platforms, and effectively push it all to the user. And at law, if the user disseminates infringing work generated by AI, then that user would often be infringing a third party's IP.

SPEAKER_03 16:28

You also mentioned risk in terms of the IP ownership of outputs.

SPEAKER_04 16:31

That's

Can AI Outputs Get IP Rights

SPEAKER_04 16:32

right. The third area getting a lot of attention has been whether IP rights subsist and can be protected in AI generated works. Again, the situation has differed somewhat across different jurisdictions. In the US, the US Copyright Office has maintained the position that there is no copyright subsisting in AI generated works on the basis that there is no human author. In the UK, there is a specific statutory provision dating back to before the current AI trend, which purports to grant a more limited form of copyright to machine generated works. But its application is uncertain, and the government recently implied that it might repeal that section due to that uncertainty, without saying whether it would replace it with anything. Equally, in respect of patent protection, successive jurisdictions have ruled that patents can't be granted in respect of AI inventions purely generated by AI, because there is no human inventor. IP officers have indicated that keeping a human in the loop may allow IP rights to subsist to the extent that the human is involved in creating or editing the work. But again, it's an area of uncertainty and it poses a real risk to organizations which rely on IP rights to protect and commercialise their work, if AI is being adopted by employees to generate those assets.

SPEAKER_03 17:46

Thanks,

Key Takeaways And Closing

SPEAKER_03 17:47

Pete. There's clearly a lot still to be determined, and we appreciate the updates. I'd also like to thank Jazz, Andrew, and Pete for joining me on cross-examining AI today. I hope listeners have found today's discussion helpful. Do subscribe and reach out if you have ideas for topics you'd like us to cover in the future. Goodbye.