Exploring AI Matters
Our mission is to help the policy community understand the breadth and richness of AI and the potential for such technologies, wisely applied, to augment all sorts of human endeavors.
Some AI tools are able to assist humans in performing tasks faster, more accurately, or more efficiently. Some, however, are inaccurate and unreliable. Who or what we hold accountable for these flaws, and what incentives we do or do not create for their correction will influence AI’s hand in how we work.
In this series we will refine, sharpen, and clarify your understanding of AI.
Exploring AI Matters
Episode 17 - Military intelligence and operations - It’s All About Scale
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
In this episode of Exploring AI Matters we have two guests from the most senior ranks of the Royal Air force, Air Marshals Philip Osborn and Edward Stringer. Both men joined the UK’s Royal Air Force in 1982. Air Marshal Stringer served 36 years until 2018; and Air Marshal Obsorn served 37 years until 2019.
Intelligence has always been about dredging through vast pools of dull bureaucratic documents, seeking illuminating insights. In this episode we will explore military thinking about how to responsibly use AI. One of our guests has said, “data is the new oil,” and expresses concern about the tainting of our data by our adversaries. We discuss with our guests the Royal Air Force’s use of AI in intelligence and operations. [2024-04-19]
Welcome to Exploring AI Matters. This podcast series, previously known as Mind the Gap, Dialogues on Artificial Intelligence, will continue to appear in the ABA series to the extent that, in addition, all of the episodes, old and new, will now appear under our new podcast name, Exploring AI Matters. Thank you.
SPEAKER_04In this episode, we will explore military thinking about how to responsibly use AI. One of our guests has said data is the new oil and expresses concern about the tainting of our data by our adversaries. Intelligence has always been about dredging through vast pools of dull bureaucratic documents, seeking to make sense of them. We hope to discuss with our guests the Royal Air Force's use of AI in intelligence and in operations. Welcome to Mind the Gap, Dialogues on Artificial Intelligence. I am Roland Trope, a national security lawyer.
SPEAKER_05And I am Charles Palmer, a computer scientist. We are your hosts for this episode of Mind the Gap Dialogues on Artificial Intelligence. In addition, we have two other hosts.
SPEAKER_06I'm Anna Adams, a national security lawyer. And I'm Mark Donner, a computer scientist.
SPEAKER_04Each episode will be led by two of us, with the others adding impromptu questions and comments as the spirit moves them. Today we have two guests from the most senior ranks of the Royal Air Force, Air Marshals Philip Osborne and Edward Stringer. Both men joined the UK's Royal Air Force in 1982. Air Marshal Stringer served 36 years until 2018. And Air Marshal Osborne served 37 years until 2019. Air Marshal Edward Jackson Stringer's overseas operation commands included service in Libya, Afghanistan, Iraq, and the Balkans. As Air Marshal, he led the Defense Academy and Joint Force Development and Joint Forces Command, where he was responsible for advancement of the operational effectiveness of the current and future joint force of the UK. Air Marshal Philip Osborne's overseas deployments included Germany and Turkey. His staff tours all had a capability focus with responsibilities for command air, carrier strike, land forces engagement, weapons, and joint enabling capabilities. In 2015, he was named Chief of Defense Intelligence with the rank of Air Marshal. Charles, why don't you start the questions?
SPEAKER_05Well, welcome to Mind Agap. Let's begin by asking you each to describe how you came to your interest in this AI thing.
SPEAKER_02Well, I think it's fair to say that it chooses you rather than you choosing it. It's something which, from my perspective, I was aware of before I became head of our military intelligence in the UK. But as I took over, probably about 2015, it was obvious that artificial intelligence had supplanted big data analytics as the new thing that everybody talked about. I think the difference is that from an AI point of view, the breadth and depth of its utility and threat was something that was pretty apparent to us from an intelligence point of view, such that we took a bit of a technological lead with other intelligence services, predominantly our colleagues in the Defense Intelligence Agency Agency in the US. Yeah, thanks, Jill.
SPEAKER_03And um I'm very sorry to start with a small correction, but um the intro 2018 was the start of my last job that you described as Director General Joint Force Development. I left in 2021. And I think my artificial intelligence interest goes back to when I was commandant of the Air Warfare Center and had the cyber cell and started to think very much about information warfare. Though if I traced it back, flying in 1995, 96 in the test bed Jaguar at Boscombe Down, flying with the first terrain elevation database and actually flying with digitized computation probably was the start of the journey. Very much like Phil, it chooses you, and for that last job, thinking about the future, you realize, and of course, with the Defense Academy, one looks at a historical context, and I think it's very important not to lose historical context and not be seduced by the tech of the day. The historical context tells you that all warfare follows the industrial revolutions, the changes in the socio economy. Americans will know this better than most, because industrial age warfare started in 1861 with the US Civil War. We're still to an extent in that paradigm. And so the shift to the information age makes you think about what does this mean? And I guess the first time I put my thoughts on paper was to the West Midlands University's Military Education Conference, annual defense lecture at Birmingham University in in 2015. And having been forced to stand up in front of a proper academic audience and say, where's warfare going, and really think about it, allowed me to coalesce those thoughts. So very much like Phil, I'd agree, in some ways, uh it selects you. I understand that.
SPEAKER_05Well, of course, you're aware how AI is advancing and leaps and bounds and has seems to be just left ahead of all of our expectations. Has your thinking about AI changed or evolved over the time? How?
SPEAKER_03Okay, well, I'll go first on that one then. Yes, because I think my my thinking on this probably started when I read decades ago about how supermarkets were just using big data and quite simple data analytics to produce answers that were, and I think this is the important, a very important phrase, counterintuitive. And so store managers would find pantechnicans of stuff arriving for a weekend that they hadn't ordered, and they would ring up and say, Why have you sent me all this stuff? But by the end of the weekend, it would have sold. And there was a realization there that we as a individuals, of course, for your listeners here, we're all remarkably individual. But you know what? It doesn't take us to blob up into forms of society, and we'll become really quite predictable. And so I thought initially artificial intelligence was essentially a way of just managing data, and therefore it was a way of, if you like, creating digital, virtual, human abacus shufflers. Then you got into thinking about artificial general intelligence, and then you started to look at neural networks, and then you get into Deep Minds, Alpha Go, and especially Alpha Zero. And if anybody's watching, just Google up Netflix, the Alpha Zero movie. If you don't know about this, it'll blow your mind. When a machine that isn't taught anything about playing Go, the most complex strategic game in the world, is just taught how to think about games and taught the basic rules, plays itself for something like nine hours, and at the end of nine hours has learnt enough in playing itself to defeat Grand Masters, then you realize you're into a different world. And so my thinking a few years back suddenly accelerated to we really do need to think about this not as a very clever, self-oriented calculating machine, but something that can come up with insights. And maybe later on we'll come back to things like chess and go and the because it does provide an interesting way of thinking about how AI is going to alter general shipping around.
SPEAKER_02Phil? I guess from from my point of view, it's really thinking about the application of AI from an intelligence function. I was involved in commissioning research into big data analytics and then into artificial intelligence before becoming CDI, but quite quickly, forcing us to think about how you take intelligence disciplines, how you apply the thinking that you would do perhaps to an all-source intelligence problem and the generation of potential futures, and really understand the opportunity that AI gives you from both the feeding of good information to your own decision makers, but also how do you counter the inevitable deception that might be run by the opposition to AI? So I guess my thinking accelerated more from a threat and opportunity perspective rather than the science and the technical. So very, very applied to how you use AI in the intelligence lane. And then that quite quickly, I think, leads you into other areas of both opportunity and threat, which I'm sure we'll talk about in the next 20-30 minutes or so.
SPEAKER_05Fascinating. How can we use it? Do the all the different ways that you could apply AI, is it really changing the way it's the military programs use it? Are we having to just modify what we have, what we do, or is it a complete rethink?
SPEAKER_02I I guess it depends where you start. There'll be some aspects where if you don't change process, then you'll lose the opportunity that AI offers. I think we have to be clear the real advances in artificial intelligence will come from the commercial side of our lives rather than the military side of our lives. That brings with it a whole host of challenges in terms of the sway that corporates, perhaps transactional corporates, have in terms of the tools by which governments and militaries make decisions. But again, we can come back to that. So I think there's something around if you don't change process, you won't manage to take the advantages that AI offer you in some lanes. But I would say, again, perhaps because of background, there is something about humans leading AI and the application of it. I think that's particularly true in a military sense, given the nature of military operations are by definition human. And therefore, I guess the answer to the question is it depends. You do need to change processes, you do need to embrace technology, but you also have to keep an eye on what's most important. And from my perspective, it's making sure that the humans are appropriately at least on the loop and perhaps involved in the authorization of activity in a way they would not have been before, why they're a bit further back intellectually, but they're still involved. Edward?
SPEAKER_03I think there are there are two levels. The first is really quick and easy to go through. AI can take over a bunch of routine processes, and it's already there, it's been there for a while, it's not scary. The complex weapon system, like a frigger or a nuclear submarine, will have AI running on it now that is finding patterns that would previously elude all but the most sensitive engineer, and you'll find out when things might fail, you can therefore predict you can get a better mess. So the fleet management side of stuff, if you like. On the other side, I I think there are some really fundamental questions about command and control. And I would look at, by analogy, all right, slightly simplified, but I'm sure we can you know tease this out later on in this broadcast. If you look at chess, for the last 10 years plus, no chess grandmaster can beat a chess computer. That's accepted. The best they can do is draw. Some grandmasters, the the more charismatic, flair-driven, have just bemoaned the fact that the computer has removed the art from all of this and almost had a group stop. Many other grandmasters have said, well, hold on. If I now form a team with me, a chess coach, a data scientist who understands what the chess computer's doing, then actually we can start to work with the computer and we can make me a better chess grandmaster when we're coming to make decisions. And for them, decisions, of course, is when they go on to into chess competitions against another human player, but they're a product of human machine teaming. And that, in a simplistic level, is posing now a very good player who's augmented by a against perhaps a charismatic intuitive genius, the Kingfisher moment from a course that Phil and I will know well, high commander staff course in the UK, which tended to suggest that generals were uniquely geniuses when it came to military performance. I don't think that's true anymore. And so when it comes to the fundamentals of the employment of military force, and your question was, does AI pose threats to conventional military platforms? Well, those platforms will change to an extent, and it could be a serious extent depending on how much we allow autonomy, which is what Phil was talking about. But I think some of the fundamentals of how we manage intelligence, knowledge, command and control are going to change very fundamentally. And I think a lot of what we've prioritized and privileged, heroic military leadership in the past, with command and control of the brilliant general, the kudoy that cuts through and sees the essence of the problem, all of that, that really is going to have to change. And I'll finish by saying if you want to see this happening at still quite an immature level, just look at what Ukraine has managed to achieve in a year, repurposing a lot of commercial tech, including elements, quite simple elements of AI, and it's created much flatter structures, much quicker structures of command and control. And fire support in Ukraine is now referred to as Uber fires. And that should really make us think because it's demonstrating that once again, the way the military employs force, and that's the nature of warfare, and that hasn't changed. The character of that, though, is following the changes in the socio economy from a centrally controlled mini-cab organization where one brain sends each cab to where it needs to go, to an Uber organization where you're now linking the various players in the market in real time to create very short notice, contracts of convenience, and that will change the way the military is commanded and controlled in a very significant way.
SPEAKER_04I I'm a little worried that our audience might not have really grasped what you have perceived to be the change in the way that Ukraine has used fire control. What did it used to be, and what are they now able to do? Because your answer merged with the commercial market. I want to make sure they understood what you were saying on the military side.
SPEAKER_03Yeah, sure. If you look at the well, I don't know how long we've got here, I'd love to do an analogy because I also think that we should be quite measured when we look at the what are considered to be the threats of AI. Because of course, in every change and every technological leap in the military sphere has brought with it shades of shock and horror and awe. If you go back to 1861, start of the US Civil War, you look at a retired military officer with not much of a massively significant career behind him, Ulysses S. Grant, but he comes back into the military in 1861. He brings with him and promotes to senior officers a bunch of people who are familiar with the rail, let's just say the railroad and the telegraph. And suddenly those people are repurposing civilian technologies of the new industrial age, allied to real changes in firepower and uniformity of factory-produced, proper artillery, not just cannons, rifles, not just muskets. And suddenly lethality has taken two steps two leagues forward. You can move by the railroad huge quantities of stores at great speed and over great range, and down every railroad is a telegraph wire, and you've got cross-theater near real-time communications. Of course, it changes the entire way that the way that warfare's fought. Skip forward 160 years, if you like 1863, 2023. And the Ukrainians are linking Elon Musk Starlink with software developed by Palantir to help businesses understand exactly how things are working and cooperate around their business more efficiently. They have knowledge from commercially sourced drones. They're going through 10,000 a month at the moment, but these things are cheap as chips, so they can afford to do it. And the whole information environment has changed from one that's more hierarchical, where all information comes into the commander's headquarters is crunched and instructions flow out, to now a much flatter structure where sensors and shooters form immediate markets. And that's why I tried to, in a very overly simplistic, I admit, analogy, linked it to your minicab firm where every single customer rings the minicab headquarters and one flustered individual tries to mix the cab with the person who's ringing up, to you now, Uber, where you call up an immediate whoever is closest by who's free says, I'll take that ride. Well, you now have a sensor who says, I found a Russian target, and the artillery system says, I can do that, I'm free, I can engage. And it doesn't go up and out, it stays flat and horizontal. And as Phil said, that does pose questions about who ultimately is making the decision that it's appropriate to engage. There are some secondary considerations, but technically, we can repurpose a heck of a lot of technology now to create much faster, flatter systems in the way that information age companies are doing, and we can apply that into the military sphere of command and control.
SPEAKER_02And I wonder if I could just come in and add a couple of comments. All Edward says is true, but I think we have to remember everything's relative. So this is a this is a comparative fight of thought and a comparative fight of action. And therefore, you're probably looking at the relative pace of decisions dominating the command conflict, i.e. the the tussle between a friendly commander and an enemy commander, but a different sense of what underpins that pace of decision making. So understanding previously, you'd want to understand whether your intelligence was high quality or not. The same is true in the kind of conflict Edward's talking about, but it's probably much more about the assurance of data. All data is biased. It's just understanding how biased your data is and how biased the opposition's data is likely to be. So there'll be some fundamental tenors, but I think what we're talking about, and again, Ukraine is a good example, is how successful militaries will embrace the speed that AI technologies offer, and they'll get a balance between assurance and pace in a better way than the opposition. And again, that that brings a whole bunch of considerations around escalation. How do you manage escalation? It brings considerations around who owns that data and the role of corporates, transnational corporates in national security activity. But I do think it comes back to the pace of decision making, either at the tactical level or in the context we're talking about at the strategic level.
SPEAKER_04When you talk about the pace of decision making and the data, I have a couple of questions to ask as follow-ups. The first is are you concerned about the pace getting so fast that at a certain point in order to stay within your adversary's decision loop, you have to delegate to the AI making certain decisions that used to be made by a human in the loop?
SPEAKER_02I think you have to be prepared for that consequence, otherwise you lose. If if you if you give the advantage of escalation to the opposition from the off, then then you'll never be able to take the offensive. So I think you have to have a number of positions where the human is comfortable to delegate decisions increasingly to. Machines, you may have machines guarding those machines. You might have AI watching over different AIs doing the work for you, but you control those steps of autonomy, and you as a human control that the parameters that your AI works within. The downside is every time you put a parameter or a constraint on a machine, you give the opposition the opportunity to do the diametric opposite and perhaps take advantage. You could think of many parallels in the physical world, but I think that's particularly true from an AI point of view.
SPEAKER_04Well, when you when you give that description, I guess one of the things that comes to mind is that civilians being what we are, we tend to think almost in terms of commanders who aren't at risk of themselves being taken out. That's of course not reality. So if you have somebody who is the key human in the loop for an AI system and they are taken out of action, does the A do the current AI military systems proceed to operate without the human's further direction, or do they wait for the human to some human to step in and regain control?
SPEAKER_02It depends what you tell the AI to do. I mean, we're talking, we're talking many ways about autonomous systems and the way that autonomous systems operate today. In in the early years of autonomous systems, they weren't autonomous. Humans were in the loop, and if you lost connectivity with normally in this case a weapon, then the weapon would go into a loop or some kind of holding pattern or crush itself somewhere safe. I think that's a something which has got to be different if you're looking to apply technology in terms of strategic, uh strategic decision making. I think we also have to bear in mind, though, that humans make some really rubbish decisions. Humans can make really bad decisions, they get angry, they get all upset. Some humans actually break the law. So there's something here about machines making decisions when you've thought through the decisions that the machine might be offered, actually should, could give you a position where they're making better decisions than humans, not just quicker decisions than humans. So I don't subscribe to human good, machine bad. I think there's a real thing here, coming back to a phrase Edward used around human-machine teaming. Edward, I'm sure you've got a comment.
SPEAKER_03Woody Machine have conducted Miley Massacre. And that's the answer whenever you get presented with the killer robots question. And yet, Phil, I wasn't going to come in actually, Phil, because I think you've you've you've made all the points brilliantly. All I would add is a bit of real-world anecdote, back in 2019 at a gathering of the futurologists of NATO at uh Norfolk, Virginia, after a bit of a vanilla presentation that said we'll always have a man in the loop. It was my brilliant US opposite number, US Marine Corps Lieutenant General himself, that went, stop, if you write that simple sentence, you've just massively limited us. And what you have to say is appropriate human control. Because if we're talking about some elements of cyber or some some elements of just hypersonic attack, if the speed of defense has to be left to some extent to the automatics, then we are almost back into going back into some of the elements of uh international humanitarian law, but only from the point of starting with the essential initial moral principles and making sure that then the assumptions that were applied when these conventions were written now eighty odd years ago still make sense in the modern world. Because would you n would you disallow a missile defence system because it was too autonomous if that meant that the missile defense system would not defend hundreds of thousands of your citizens, maybe millions? And those are the questions we have to think through.
SPEAKER_04Well, in terms of what again the civilian world tends to ask in the way of questions such as what threats does AI pose to conventional military platforms like aircraft tanks and ships, it sounds like a better question to be asking is what kinds of concern worry you the most with regards to AI in its military applications, rather than asking what threats does it pose? Because from what you've described, it may be figuring out where the appropriate level of control should be, or it may on the other side be has an a pure adversary now developed a level of speed in its use of AI that exceeds ours, and we have to change our doctrine. So is it fair, to the extent you're allowed to answer, to ask you where are you most concerned about AI in its use in military applications right now? And I realize that could also be that maybe it's not good enough yet to satisfy you from a defense perspective.
SPEAKER_02From my perspective, the thing that would concern me is if the opposition get better at it than we do, quicker than we do. It's simple as that. It's interesting we talked about the way that we might ensure that our use of AI is appropriate. We need to make sure that the opposition are following the same rules. There's something here about nuclear arms control and the parallels or not that may be achievable. But fundamentally, if we're talking about technology that speeds up decision making, that lays out possible futures to aid that decision making, all at the machine of at the speed of a machine, that actually can manipulate, if applied properly, the kind of picture that you are seeing that informs your decision making, that maybe manipulate the data that you're seeing, then you have to maintain a level of parity. Otherwise, your your balance of deterrence has fundamentally failed, I would offer. So the threat, I think, from an AI point of view is less how we would use it and how would we misuse it. That is a challenge. But the the primary threat is the opposition gaining an advantage, which it would be difficult to recover.
SPEAKER_06This reminds me of the debates from probably the 1970s on a doctrine called fire on warning that that people struggled with for many years and as near as I can tell, never quite figured out what to do about.
SPEAKER_03Yeah, I could I just add to um what Phil said. And I think the the the threat at the moment is things are moving so quickly that two things. One, it's hard to keep abreast of where we collectively are, and two, the fantastic breakthrough, the amazing breakthrough could be made by the sort of player that previously wouldn't have made such a military breakthrough because it was always going to be made by a superpower. At the start of the year, to train a sort of foundational model for AI took about $10 million. By the time I spoke to a friend of mine three or four weeks ago, because I was giving you a presentation, I just wanted to check up on something. I'm sure he'd said that had come down to $1,000. And when I rang him up, he said, I've just got a Stanford paper here. This is now two weeks ago. Times like two weeks matter. I've got a Stamper paper from two weeks ago showing that they managed to train a model on 600 bucks, and they reckon if they've had longer to plan how to train, it would have been less than that. So $10 million down to 600 from January to June. That puts the ability to run these things into pretty much anybody's, if you like, uh, resource bucket. And I think therefore, it we don't know where the next big breakthrough is going to come from. I think there are some advantage that might come to later on in the podcast in that for the for the Western Alliances. But for me, it's the speed at which things are moving and the fact that the next breakthrough could come from a very small but brilliant research outfit somewhere, and we don't know who that brilliant but small research outfit is going to work with.
SPEAKER_04I'm surprised that you haven't included in your concerns. Again, this is my lack of knowledge, perhaps speaking, that one of that one of your major concerns isn't protecting the integrity of the data either at the training level or subsequent tweaks to the model. In other words, I would have thought a peer adversary, as one of its ways of slowing down your development, would be to contaminate the AI system itself. Is that really not is that an illusory threat that I'm asking about?
SPEAKER_03Phil, do you want to go or should do you mind if I dive in? Dive in, why not? I think it's very important to look at how you would look at contaminating data that people use. And I know I've known for several years now that lots of companies are looking at this. But data is now being stored in many, many, many places. If you look at what uh Zelensky has said in Ukraine quite quietly, somewhere, it didn't hit the headlines. We talked about the most important partners Ukraine has got international partners. He said first is the US government, the second is Microsoft and Amazon Web Services, and the third is the UK government. And that just shows you the importance of, and Phil mentioned this earlier on, you know, the power of these now huge, big multinational companies, in this case, working benignly into Ukraine to extract and protect their data, which a quite competent policy, admitting that trying to understand the Russian policy is tricky, but a policy like Russia had found has found itself pretty much unable to get at or tarnish in a significant way. And of course, data can be replaced. There will be ways of storing it, and now we're training models on synthetic data. So my personal sense, and it is only a sense at the moment, and this is why I talked about the new oil, just as oil has become ubiquitous, and every industrial age war has to some extent or other, it's become protracted, had an oil plan to try and target the oil production and distribution of the opposition because all platforms were hydrocarbon dependent and never never quite managed it. Of course, it could be operationally significant, but never quite managed it. And you will know that if you break down the middle of the Sahara, you'll find somebody who comes out of a Bedouin tent with two jerry cans and will sell you some diesel. And my my sense that data is always going is is is always going to be available. Yes, there will be some twos and froze. If I'll try and corrupt your data, I'll try and corrupt this. There'll be some form of algorithmic warfare, data contamination warfare. But my suspicion is it's never going to be massively decisive one way or another. But that's purely a personal perception. Phil might have a different one.
SPEAKER_02So I would add to that two things, I suppose. The first is I'm less worried about data assurance because it's hygiene. It's a thing you've got to do. And in the consequences in the context Ed was describing, if you're using massive amounts of data, then actually you're probably more worried about the inherent bias within those data sources, yeah, i.e. unintended contamination rather than intended contamination. But what I would offer is I I do think there's a threat in where you're working in absolute real time on the collection of data, and those really immediate data points are changing decisions. You'd need to have a lot of them, but but those immediate data points are changing decisions, then there may be the opportunity for the opposition's AI to be showing you what you what they want you to see rather than what's actually happening. I think that's a different thing. I think there's a there's a thing about data contamination and there's a thing about deception. And I think we have to keep them, keep them separate. But data assurance of the vast quantities of data you've using is a hygiene factor and therefore manageable from my perspective.
SPEAKER_04Charles, I have a another question, but I think from what they were just saying, we should come back to my question and go to your data quality question since that immediately leads from what they were just saying.
SPEAKER_05Okay. So businesses are all very concerned about this data as the new oil kind of thing. It's just to keep up with uh competitors or to stay ahead of other issues like regulation. Uh they really have to depend on the data. And as you said, it's a it's a hygiene issue. But the way I'm looking at it, it's it's a whole lot more data faster than we've ever dealt with before. Some of our other podcasts approach this, things like dealing with data from the new telescopes or climate simulations and so on. So how do we make sure? Is there anything unique about the data for AIs, their speed or their the lack of tolerance for noise in the data that we have to worry about?
SPEAKER_02Always interesting when a computer scientist asks aircrew a question around data and computers. And I don't know about you, Edward, but it makes me feel very, very under pressure because I suspect Charles knows the answer to this question, but we'll press on anyway. And so, from one aircrew dude's perspective, the you know, from an AI point of view, more data is good. For years, militaries have had far more data than they can possibly analyze. Most intelligence analysts analyze are able only able to analyse the data that's in front of them rather than the data that's theoretically available. I think we're seeing some of the advantage that comes with being able to look at the data that's available when we talk about open source intelligence. And that and that, if you like, is is less a reflection of what intelligence is openly available and more the quantity of intelligence that's or of data points that are available. So again, I suppose I would turn it on its head. The one of the huge opportunities of AI is it allows you to manipulate data in a way that you could not possibly have consumed it before. It allows you to make sense of data that you wouldn't have even perceived was there, let alone had time to analyze it. So I guess I'm less worried about it, about the prevalence of data because we know it's there. Now, how you get it, how you assure it, how you brigade it, how you make sure that you have different types of data available to you at different times. Again, I'm not underplaying the challenge, but it's a technical challenge that can be managed. At the moment, militaries have far more data than they can possibly analyze, and even starting to make inroads in that has got to be a great thing.
SPEAKER_03Yeah, I I'm I'm I'm sanguine about this because it's all trying to remember five years ago, maybe a bit more. Um remember taking a general down to a company that was looking at this and he started asking these questions, and this company owner immediately shot back with what you're talking about. Messy data, what we used to worry about formats a heck of a lot, and it's still important for ease of doing stuff. But my sense is as tech advances, we're becoming much better at just being able to cope with elements. I think, as Phil said, the problem will be if people want to interrupt with some form of deliberately introduced bias that gets into a loop as previous data is churned through, you know, whatever the the son of the son of chat GPT da, and that starts to become a contaminated uh a contaminated stream because we got happening. But at the moment, no, I don't see it as a problem. I think the tech is starting to help us by being able to read multiple, you know, not just formats, but multiple however the data is stored, it's it's storing a is it image, is it audio, is it written text? You know, what actually now we're being able to crunch that automatically in a way that even only a couple of years ago wasn't possible. So I'm I'm not as worried about that until someone comes and tells me they've got the measure or the countermeasure that turns that on its head. And then there'll be the inverse.
SPEAKER_05Yeah, the reason I I raised that question is um the GPS challenge, which is we depend on this data, we depend on it being available and reliable, and yet we know there are entities who will jam it, which is pretty obvious what's going on, or spoof it. And there's a case where we're depending on the data and somebody has learned how to manipulate it. So I guess until it becomes real time like that, we'll just have to have to watch that.
SPEAKER_02I agree, but the essence of what we're talking about, you haven't got an AI if you haven't got a huge amount of data. And to have contamination across that huge data set other than bias, because we know that all data is biased, but to have contamination against the whole data set is a is a really interesting thing to think about. You know, there there are there are arguably more data out there that is true or close to the truth than data that is not. So so I think the you know, by definition, that was an article of faith, I guess. By definition, if you're talking about an AI, you're talking about a shed load of data, and all of that data being contaminated, I think, is unlikely. Yeah. Okay.
SPEAKER_05So shifting a little bit in our earlier conversations, you mentioned something about protecting the digital weight. I've heard a lot of terms, but that one was new to me. Could you expand on that?
SPEAKER_03Well, I think that was me. And I don't say too much, give too much away, except that well, let's cut to the chat. Look at what's happening now in Ukraine, where the Ukrainians are Ukrainians are extremely good at exploiting social media to find individuals who have posted stuff online about what they've done. That is part of a digital wake. You could then look at the absence of information and say if someone suddenly goes offline, where are they? What are they doing? And previously it would have been impossible to track all these folks, but as Phil said, AI machine learning depends on huge amounts of data to function, and suddenly tracking the dull, banal minutiae of life is possible. So I'll probably end on a sort of open source, positive note with I think I think, by the way, from memory, Adrian Zens, who spent a lot of his the last years of his life tracking down from the dark corners of various internets the document the banal documentation of an autocracy, in this case the Chinese one. And if you're a low to middle ranking Chinese bureaucrat in Xinjiang province, you will want to make the best of what you've done in the last reporting period. How much have you spent? How much have you managed to accomplish? Where are you on your set? Blah, blah, blah. So all everybody leaves now behind them additional weight, which previously would probably have gone unnoticed. And the Zens has done in Xinjiang Province. If you can stitch together and track down who's built what, how many building materials, which uh, how many prisons have been built, how many people per prison, what's the catering import divided by X per person per day? Rather implies there must be 120,000 people in that particular facility. You can build up a big picture from banal bureaucratic information. Previously, we would not have had a date the size of Rhode Island of a hangar full of Mandarin-speaking analysts who could have plowed through such banal bureaucratic stuff, but now you've got machines that can do it. And so the digital wake of just banal institutional or even individual information can now be pieced together to develop insights that are massively revealing in a way that was just simply impossible before.
SPEAKER_05Excuse me, Brollen. Sort of the a spike in pizza delivery orders around the White House indicates uh something's up.
SPEAKER_03Well, that's why the that's why the Chinese break into the uh the office of some like I think it was called the Office of Personnel Management in the US a few years ago. Phil will know more. Phil, shall I hand us? to you because I'm sure you actually studied this with your responsibilities in your last job.
SPEAKER_02No, I I don't disagree with what Edward says. I think that we have to bear in mind that the digital wake in the modern world is inevitable. So what you're what you need to have at best is or at least is an awareness of the digital wake that you're generating. And there'll be some context where you make a judgment it doesn't matter and then there'll be other contexts where actually you do need to make a judgment that it matters. That's a normal decision-making process for individuals. I think the issue that Edward's highlighting is you have to have an awareness of the digital weight of organizations and groups of individuals at the if you like at the at the national security level to then understand the behaviors that you're portraying and perhaps the signals and the indicators that you need to modify. You can't remove a digital weight you have to be aware of it and at times counter it and modify it with an interdeception.
SPEAKER_04It may be a trivial example but I think there was a a reportage a couple of years ago of a running application that many US military had on their digital watches or exercise wristbands that you would apparently access and see where they went on their daily runs, which is in probably a small sense a security risk, but you it it exemplifies I think the kind of digital wake you're talking about that previously you wouldn't have spent much resources trying to gather. So it seems that you've highlighted one way in which AI changes the practice of intelligence gathering is there any way in which AI otherwise changes the practice of intelligence gathering or analysis besides the one that you just mentioned?
SPEAKER_02So what AI should do is allow the analyst to analyze rather than data collate. So available information should be presented for analysis and in then a lot of that analysis should be done by the machine such that your analyst is probably overseeing process rather than doing the analysis itself. I think we also have to remember that analysis, intelligence analysis is all about uncertainty. You can never be certain about the future and therefore good intelligence and analytics lays out prospective futures and the indicators and warnings that you would see against those futures. A machine should be able to do that in a way that doesn't just focus on one or two outcomes normally constrained by the breadth and depth of the human mind but can lay out thousands of different outcomes and the indicators and warnings that go with them and then as you progress through time indicate which of those outcomes are more likely and then probably modify. So I think there's something here about detailed analysis and the management of uncertainty that AI should be able to help intelligence analysts and therefore decision makers in a way that even the most talented analyst would not be would not be capable of even if they could see all of the data that's available and it's not it's just not humanly possible.
SPEAKER_04Can I ask a question more on the that makes us a little more hardware rather than just data in another episode John Blackburn, the former deputy chief of the Royal Australian Air Force observed that a significant challenge for the use of AI by Allied Air Forces is how to integrate first generation AI into fifth or sixth generation combat aircraft.
SPEAKER_03How do you think that challenge might be addressed if you've had an opportunity to think about it as RAF officers okay I was in Australia recently and Phil lived there for a bit so he's I'll give him some time to think I think there's a small category area here talking about first generation I and almost conflating that with fifth or sixth generation aircraft they're not the same thing. We're not talking about the same generation it's not as if Generation 1 AI and sixth generation aircraft AI has come into its own in the last few years as and our fifth generation aircraft were being developed 20 years ago. So I I I'm not sure that the or rather I think the language used betrays a slightly false prospectus here much as I genuinely love the thinking of the role of the Royal Australian Air Force. I I would say the there is a problem here is that even fifth generation aircraft were not built for the information age without going into without going into too much detail. And therefore you could easily put that sort of software that we spoke about on any platform that analyzes does fleet management predicts when in 200 hours that alternator's going to fail because the frequencies and this that and the other indicate that some bearing is about to go and all that sort of malarkey can you know you can do all that but most of our platforms that are still in service were still constructed in an era where as Phil said earlier data was a cost and most militaries want to get rid of it. I certainly know that one of the most advanced frigates that the Royal Navy has the data is owned by the manufacturer because the Navy didn't want the responsibility of storing it and most of the data was considered to be to do with reliability and therefore given the maintenance contracts why not leave all the data with the manufacturer and it might make its maintenance better rather than thinking through as we're now talking about even what looks like Coffin's noise pulled in from various sensors if I can now correlate that with the sensors of 400 4000 4000 other from simple sensors to quite sophisticated platforms I might be able to distill signals from what was previously Kacoffin's noise and so I would I I I would say there isn't actually a problem with integrating AI into those platforms per se. The problem is has we have we got the architecture that will take all the data that something like an F35 can hoover up per second given the sensors on the platform and then go and do something with it if you put it into something which I will very simplistically call combat cloud and within that con connectivity then start as I say finding signals in what was previously noise.
SPEAKER_02Phil so if if what the question means is how do you get a hyperintelligent organism on a fast jet somebody make capable of making fantastic decisions as a as a Royal Air Force tornado navigator I'd say we had those on the tornado for years they were called navigators moving on the issue for me will be not quick enough sixth generation combat aircraft should be truly integrated with data sources. Should be designed to operate within an AI enabled construct fifth generation or not so actually the technical challenge I would offer in in the relative near term five, ten years or so is less getting AI onto the platform and more getting AI enabled intelligence knowledge decision making whatever wrapped around the platform so you you have a you have a system Edward referred to it as a combat cloud you have a system within which the aircraft is just one component and that system is fully AI enabled almost the last thing to catch up will be the platforms within that system of system and the most complex platforms within that system because they're the most difficult to change technically so I think it's less about integrating AI onto fifth or sixth generation combat aircraft the benefits would be relatively tactical and more about the challenge of getting AI to be a positive contributor to the system within which those platforms sit down I'd be interested to know if you could address the following question.
SPEAKER_04You've been talking about the adoption of AI presumably with mostly a UK perspective but you've given us some comments on Australia and the US is there a significant difference in the adoption of AI by different militaries and if so to the extent you can can you contrast them and when I say different militaries I don't necessarily mean by the same size. I would think there'd be drastic differences between Ukraine Russia Israel China Australia but I and I don't know if you if there's even a way of grouping those but it would be interested to know if all of them tend to do it in roughly the same way or if there's quite divergent ways and what the advantages are that some seem to be gaining by the way that they are adopting and implementing it.
SPEAKER_02I would offer the two major differences between different nations will be the firmness of their intent and their risk appetite. So if you're really clear that you want to secure the advantages of artificial intelligence and your risk appetite is low around error or miscalculation then that puts you at strategic advantage. I mean you may come off worse because you've made a significant miscalculation but that puts you at a significant advantage. So I think if you look at different militaries it will come down to a combination of how firm is their intent, how much effort, how many how much resources are they willing to expend, probably at the cost of other activities and what's their risk appetite how how willing are they to run the risk of miscalculation that inherently puts Western militaries at a disadvantage now whether that's the right thing I think most of us will argue that is the right thing but you have to accept that puts Western militaries at a disadvantage and then get on with countering that that challenge and that disadvantage let me say where we might have an advantage because I absolutely accept what what Phil has said and it's it is intuitive that unencumbered by the external and internal lawfare and regulation autocracies especially China which is worrying should be able to move quite quickly maybe ruthlessly I remember five years ago talking to uh Dr.
SPEAKER_03Robert Hercock who's an international expert on AI very much a transatlantic figure. And I asked him well I thought it was quite a stupid simple question does Chinese AI work like British AI? And he said interestingly no it doesn't so there are national characteristics in the way things programmed. Now I think all of us intuit as it's been presented that artificial intelligence finds the fundamental truth. Phil's already said all data is biased. I'm going to suggest all algorithmic writing is biased to a certain extent autocracies therefore suffer I think two problems. One they work completely independently and two even within that independence unless there's some form of enlightened freedom and there has been in the past I'm not stupid on this they tend to trammel thought if you look around what you might call the Western coalition and for the reasons I outlined earlier look at the costs now that are coming down massively in just training a foundational AI model then it is likely that a big advance could come out of a small ally let's say one we don't immediately understand like Israel but you know why not on Estonia which has had a digitally focused society for a long time or Poland. And if that were shared around the alliance we've now got Rob Hercock to be believed yet the NATO alliance has 30 different cultures working on AI in slightly different ways all of whom might make an advance and we've got the two big adversaries Russia and China through Iranian as well working very much on their own so I think there if we were to harness that properly I think there are huge benefits there and I think alluding to questions you started off with what are the dangers of AI actually dangers and risks are normally best spotted by your friends who are looking from the outside and aren't as carried away with your own brilliance as you are and that's why I'll yeah I'll finish this answer with I was very much supportive of what Prime Minister Sunate was saying to President Biden the other day about thinking through how we might regulate AI. On the face of it regulate AI sounds like oh my god we're going to trample ourselves and we're going to get backwards if we think about it sensibly we could actually use the G7 the D10 NATO AUKUS the quad to both rapidly do experimentation in I but also rapidly do the thinking on and mutual self-checking on where could this go? How could it trip us up? And so I would call for an approach to the democracies in making sure we emerge from the information age revolution at the forefront of it and don't just nervously retreat into overregulation so politically we can demonstrate to our populace we've been very cautious and very safe because I think that would be to deny the benefits that legal democracies can bring to this equation.
SPEAKER_04You know your answer makes me feel that the next question I was going to ask is ill-conceived. I was going to ask does the adoption of AI by the world's militaries make the world safer or less safe? You've given a much more nuanced basis than I would have thought there I think what I'd rather ask you then is here you are two very senior experienced officers who until recently were getting constant access to the developments of AI in the military how do you outside of the military now keep up with AI developments and keep yourself as attuned to its developments as you are demonstrating in this conversation?
SPEAKER_03Yeah it's difficult I will say it's one of the think tanks that I'm now associated with I recommended to them that we uh because they publish a weekly agenda with what's going on in the world and a few thoughts and it's an advertisement advertisement for the things they've done that we should once every two or three weeks have a almost impersonal very much like saying the economist does just rack and stack this has happened in the world of tech and saying that things change now in two or three weeks but also those changes aren't necessarily well they're certainly not happening within government structures. And that was part of the reason for for suggesting this for an organization whose product is read very widely across government it was to jolt what you would refer to in America as lawmakers you know legislators into an understanding of just how quickly this stuff is moving. And you know if you happen to have read great books you know like AI superpowers that came out a couple of years ago and you think therefore you're up to speed if you haven't really been watching what's been happening the last two months you're not going to be and it is difficult to keep up with it. And and I think therefore the I would phrase the question I'll hand over to Phil because I know he's wrestled with this as well it's how you keep up public sector structures which actually tend to be quite cautious and slow moving and move at the speed of consensus with stuff that's now being driven in the world of high tech startups and academia and is moving at a pace that I don't think we've really seen previous technological revolutions move at. Phil would you buy some of that?
SPEAKER_02Yeah I I guess I would I would I would add a couple of three things huge breakthroughs in terms of artificial intelligence and their application in the national security space won't come from militaries. Those technology breakthroughs those advancements in technological thinking will come from the civilian sector uh it you know it so so if you like that puts militaries at a disadvantage because what they're seeking to do is how do they make the best use of things that are coming to light or coming to fruition from a civilian point of view. That kind of turns on its head the things that some militaries have been used to in the last you know 30, 40, 50 years, nuclear power being an example of that I suppose. I would also probably change the question around actually the challenges how do you make sure militaries keep up with advantages in AI? I've not met many many military people who have the opportunity to sit around and think about these problems because they're all really busy. Therefore I think it's a debate like this that will inform the other way as well as the things that we hear our military leaders and national security professionals talk about inform what we think. The fundamental issue for me is advances in AI are more likely to come from the civilian world the corporate world and the challenge is how do you translate those into national security outcomes rather than the other way around.
SPEAKER_06Mark you turned on your camera. Oh yes I'm I'm this is a point of order we are running long and while I find this incredibly fascinating and I think your point is exactly right the the the everyday practitioners are busy as hell with just you know keeping 5700 balls in the air at once and you know it's it's folks like uh the two of you who have had the benefit of many years of actual real operational and practical experience who are now reflecting on the broader pictures that's a set of insights that will be of immense value to the larger community that in fact we are hoping to inform that's why we're doing this this podcast. So you're you're you put your finger precisely on the right topic that you know why why why this is the right conversation. But we are going to run out of time well we can we can go on for hours but I think the listeners will after all stop press the stop button after a while they're also six hours later than we are and we should respect that.
SPEAKER_04But I would like to just add two quick comments. The first is it has been a challenge to keep up with how much you have changed the way we've looked at AI. And as a result we had to reconsider our questions accordingly and we hope we kept them enough up to date to make the conversation as interesting for you as it has been for us. But we are deeply grateful to you and I would hope that maybe six months from now we could have another session with you. You don't have to commit to do so on the air but we would love to have the opportunity to revisit these subjects with you given what at the pace that you're describing it things will have happened in those six months will put us in a different frame for asking and having this conversation.
SPEAKER_02Roland it would be a pleasure it's uh it's it's been really really informative from certainly from my perspective to to be able to spend the time talking about these things in the way that we have so it'd be a pleasure to come back in six months. I'd check your ratings first though and you you you might find it's not a it's not a good decision from a from an audience perspective but perhaps we'll see.
SPEAKER_04Well we've we've learned that having really interesting speakers drives our ratings and we're not a ratings driven production. We are after a quality conversation and conversation that's illuminating interesting and responsive not canned not you know ignoring the issues that the question poses it's rare and it's one of the reasons why I think the four of us have invested as much time as we have to prepare for each of these because we find them exhilarating opportunities that we don't have in our daily practice at least for the lawyers Ama and I just don't have time to have these kinds of conversations or the opportunity and we're deeply grateful.
SPEAKER_03Okay well thank you very much that's that that is um really very kind of you and all I'd say is you you mentioned things moving quickly as a timid skier myself I long worked out long ago worked out that it's as the slope's steepness starts to increase that one's sense of leaning back and trying to wish it would all stop also increases. And if you can see just in front of you a shallowing out you can actually commit can't you to that slope because you can see it's a finite period. It's when the brow is in front of you and just seems to be doing nothing but accelerating away from you. And that's why I sense you've seen so many people with AI suddenly get cold feet recently because they know better than most just what the possibilities are of this and they think things are accelerating so fast that even they can't keep up with it. You know my my final thought to all your folk listening is I think we all need to get involved this whether you're a lawyer whether you're a medic whether you're a military person the speed of advance in AI is coming to change your world and you need to go and think about it.
SPEAKER_04Well and sitting back on your skis is not a way to stay on them.
SPEAKER_03No exactly that and I have the bruises and the broken ribs to prove but uh absolutely fascinating thank you both very much for your time both today and in preparing for this question and answer section and um hey I'm looking forward to doing this again.
SPEAKER_04Well thank you again thank you very much thank you we thank the business law section of the American Bar Association for their generous sponsorship of the production of this podcast we welcome questions and comments from listeners. Send email to comments at mindthegapdialogues dot com we read all comments and questions and we'll try to respond in the letters section of a future episode if you are writing about a particular episode please do mention a specific episode number please also do include pronunciation tips to help us properly say your name when we reply in a subsequent episode see you next time on Mind the Gap Dialogues on AI Thank you for listening to the Aviate Business Law section's podcast series to the extent that the section offers a robust collection of content.
SPEAKER_00To explore more about this topic or to learn about joining the section visit anbar.org slash bizlaw that's e I z