Exploring AI Matters
Our mission is to help the policy community understand the breadth and richness of AI and the potential for such technologies, wisely applied, to augment all sorts of human endeavors.
Some AI tools are able to assist humans in performing tasks faster, more accurately, or more efficiently. Some, however, are inaccurate and unreliable. Who or what we hold accountable for these flaws, and what incentives we do or do not create for their correction will influence AI’s hand in how we work.
In this series we will refine, sharpen, and clarify your understanding of AI.
Exploring AI Matters
Episode 15 - Whither the Weather Wherever
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Send us comments and questions.
In this episode of Exploring AI Matters we discuss US Air Force funded weather research at MIT with Major (now Lieutenant Colonel) Andrew Bowne, the chief legal counsel of the Air Force-MIT AI Accelerator. Major Bowne is a Judge Advocate in the US Air Force. In addition to his extensive legal credentials, Major Bowne is a PhD candidate in Artificial Intelligence at the University of Adelaide in Australia. (Since recording this episode Andrew Bowne has defended his dissertation and received his doctorate.) [2023-12-29]
Welcome to Exploring AI Matters. This podcast series, previously known as Mind the Gap, Dialogues on Artificial Intelligence, will continue to appear in the ABA series to the extent that. In addition, all of the episodes, old and new, will now appear under our new podcast name, Exploring AI Matters. Thank you.
SPEAKER_05Modern computing theory and practice emerged from Western war efforts during the years from 1939 to 1945. One challenge not solved during the war was weather prediction, which remained an art, slow, manual, and unreliable. One of the first projects funded by the Army Air Corps at the Institute for Advanced Studies at Princeton, where an early computer was under construction, was an effort initiated by John von Neumann in 1946 for weather prediction. Predicting the weather continues to be of immense economic and strategic value. Detailed knowledge of current weather conditions in many places around the world holds a particular value for the U.S. Air Force. Hello, I am Charles Palmer, a computer scientist.
SPEAKER_03And I am Roland Trope, a national security lawyer. We are your hosts for this episode of Mind the Gap Dialogues on Artificial Intelligence.
SPEAKER_05In addition, we have two more hosts. Hello, I'm Ama Adams, a national security lawyer. And I am Mark Donner, a computer scientist. Each episode will be led by two of us, with the others adding impromptu questions and comments as the spirit moves them.
SPEAKER_03Today we are delighted to discuss U.S. Air Force funded weather research at MIT with Major Andrew Baume, the Chief Legal Counsel of the Air Force MIT AI Accelerator. Major Baume is a judge advocate in the U.S. Air Force. In addition to his extensive legal credentials, Major Baume is a PhD candidate in artificial intelligence at the University of Adelaide in Australia. The views presented are those of the speaker and do not necessarily represent those of the Department of the Air Force. Andy, since you've said we can call you that, could you tell us why the Air Force is so interested in weather?
SPEAKER_00Sure, Roland. So imagine how different the world would be if military's throughout history were able to predict the weather better. How would that affect it? The D-Day landings or the Dunkirk evacuations. And of course, it's still a major factor from really tactical decisions on when to fly or not to fly, the operational decisions that affect entire theaters, even strategic decisions as we're seeing now with climate change. So for the Air Force, weather modeling is using just a vast variety of different mission areas. Aviation weather obviously comes to mind, and that comes into play for forecasting whether it's at the terminal or in route. Humanitarian disaster response applications like monitoring of uh in and forecasting fires, um, maritime applications, uh, tracking severe weather for public safety, combat search and rescue, space launch, the list just goes on and on. So weather's really always been important to military operations, and it continues to be uh a very important factor today.
SPEAKER_03Well, something that's been puzzling me is we've been thinking of this episode. I remember when the first weather satellites went up, and I think a lot of us think, well, if satellites can photograph small objects on the ground, monitor what's going on in cities, why is it they can't just tell us what the weather is? What why do we need to forecast it when the satellites are up there looking at it?
SPEAKER_00So that that's a really challenging problem. And it it does seem to be something that uh would be easy to do, but there's there's a couple of factors that that make it difficult. We are all familiar with weather forecasting. The concept that you just talked about, Roland, is what we call weather now casting. It's it's really how do you get that short-term on-the-spot identification of what the weather patterns are going to look like in the place that you're concerned about. Uh, and so what I just described, or really the goal, there's various levels of and layers to that problem. One is the type of data that we're receiving. It's is coming from all sorts of different sensors. Some of them are ground-based, like radar sensors, uh, and some sometimes radar is going to operate in other locations. Uh it could be satellite-based. And where we get that weather and where those sensors are actually pulling data from is not actually as comprehensive as many people might believe. And so there's areas of the world that just lack those that sensor data. And I'll remind everybody, I'm not a meteorologist, I'm I'm the legal counsel for for this research team, but I do get a chance to work with some fantastic meteorologists, both at the Air Force and then at MIT and Lincoln Laboratory. And so, what I've learned from them is that the the problems that that meteorologists have are often from the data, but it's also weather is just a hard problem. It's unpredictable, and we're trying to predict it. And so when we're trying to forecast it, it's going to be a challenge. Uh, we we see that all the time. We we take for granted the fact that we get a pretty accurate depiction, even as far as like 10 days or two weeks out on many weather models. And usually the best we could do is complain about it if they if they miss it, if the forecast missed uh you know a couple of inches of rain here or there, uh maybe it was a little bit warmer than than the the weatherman said it was going to be on the local news or on the the your app on your cell phone. But as uh as was alluded to earlier, that was a that's a relatively recent phenomenon that we're able to have any indication of what the weather is going to be. And now we're trying to get to the point where we could actually understand what the weather is now in a given location.
SPEAKER_03As I'm listening to you, I'm beginning to wonder which kind of weather are we talking about? Are we talking about weather on the ground that the Air Force is interested in, whether at you know, mission altitude, whatever that happens to be? What kind of weather is the Air Force really interested in?
SPEAKER_00Yeah, all of it, and for different reasons. So the Air Force has a very wide mission area. Uh, and so we have things that operate on the ground and we have things operate in the air. And with the Department of the Air Force, we also have the Space Force. And so that's a whole nother factor. And all of those mission areas have different and distinct weather problems that need to be addressed. And so, even in flight, yeah, of course, we're going to want to understand what the weather is at various altitudes, not just uh at a particular one. Uh, and the mission may call for for a pilot to operate on various uh ceiling levels of of altitude, but we also want to understand what what it's going to be like, where they're going to land. That's important. Or if there's an airdrop or other mission necessities that require some sort of activity by the air crew in the air, where they're at that point, that weather is going to matter. It's going to matter what type of equipment is used, how the you know, a say a C17 or basically one of our large cargo planes, how we're going to set up the payload, what we're going to actually use on the plane, that weather impacts all those decisions.
SPEAKER_03That that's really helpful. I I should mention that uh there's recently been a flight that to from New York to New Zealand, which because they misjudged and mismodeled the uh headwinds, they're having to take fewer people, and this was something they thought they had nailed down.
SPEAKER_00So now we put the context it affects like fuel consumption and approach and how long you're in the air. So it's it's a it's a uh a big problem.
SPEAKER_03Modeling and prediction of weather has been an interest of both computer scientists and defense department thinkers for at least three quarters of a century. Can you give us a sort of an overview as to what progress has been made at that time and what progress you think we might be making in the in the near future? Call it foreseeable or forecasting.
SPEAKER_00So the really big uh noteworthy advancements are coming from what we call hard hardware-based solutions. And so radar and satellites are are really making up the majority of the sensor data that we're getting in that time that you just described. And so, in those decades, and since radar uh has been able to be uh used by meteorologists, and then later on, as satellite whether satellites have been able to be used, and we're starting to see really unique ways that satellites are being used as well for weather and for climate also, that's going to be the state of the art for weather for many decades. Recently, we're starting to be able to apply artificial intelligence applications to helping model uh weather. We talk about weather prediction and weather forecasting. Well, what does machine learning do really, really well? It helps us predict based on the data that it experiences. So basically looking in the past at what patterns and identifying patterns or outliers and being able to uh extrapolate upon that that historic data to predict uh what we might see uh later on. Uh, and so while hardware really dominated a lot of the advancements in the the you know several decades uh before now, we're starting to see algorithmic advancements that are really starting to make uh meteorology a very relevant and and very cool field of study at this point.
SPEAKER_03Can you give us a sense as to what the improvements have been? In other words, are you able to forecast more accurately over longer periods, or are you able to do the now casting more accurately?
SPEAKER_00So a little bit of both. Uh and a lot of that comes down to the types of models that we're using, the types of really powerful neural networks that are being used uh in order to uh take all that data and come up with algorithms that are going to be really good at predicting, both for longer term and also for the very short term, uh, which is really hard to do. But there's also just collecting data. And as I said before, a lot of the data that that we have historically relied on has come from sensors that are disproportionately placed in areas of really kind of urban centers or highly populated areas or or you know, kind of the the more affluent countries uh are going to have weather radar and have the satellites that are going to be providing that really rich data in the format that it needs to be in. So, including like the time series that you could actually be able to come up with a uh algorithmic prediction and a model for that forecast, where we're now able to start looking through that sensor data and and piece it together with other existing sensors in something that's called a global synthetic weather radar. And so that's really kind of the the near-term advancement is being able to have basically fill in the blanks where we're missing an actual sensor and predict what is going to be there using synthetic radar, using basically multimodal sensor data. So other other signal um that are able to be received from from various sensors can be used to support the the predictions that that these models are making.
SPEAKER_05This is fascinating. I mean, weather modeling is one of the toughest applications uh we've ever tried with computers. And you you've mentioned AI already and its application. One of the keys to AI is of course um that data thing that you keep mentioning. How are how are you obtaining the data? And I assume it's mostly public data. Uh there's probably some fun stuff in there too. How do you know if it's any good?
SPEAKER_00Yeah, so a couple a couple parts of that question. So it is public data. So a little background on my organization. It's uh as you heard in the intro, it is a mouthful. It's the Department of the Air Force, Massachusetts Institute of Technology, Artificial Intelligence Accelerator. Essentially, what that is, is the collaboration between the Department of the Air Force and MIT. Uh, and through that partnership also with Lincoln Lab, we we kind of get from both angles the fact that they're a federally funded research and development center as well as an extension of the MIT campus. And so that that kind of three-prong collaboration between the Air Force, MIT, and Lincoln Lab is what makes up the AI accelerator. And it is we we focus on fundamental research, uh, which essentially means it's research for the public good. It's not uh going to be basically it's RD that isn't going to be restricted by ITAR and other export control act. And so the whole idea is that it is public, it's it's research for the public good, and it's research that is is going to be made publicly available, at least that's the intent from the outset. And so we're able to work with a multinational team at MIT, which is really unique for us in the Air Force. And because of that, we really do focus on public data sets. There's ways for us to work on non-public data sets as well, but for the most part, and for this uh particular project, we we do work with public data. Part of the project is going to be using existing data that we go out and collect and then curate and clean and ensure that it's something that we're able to actually do research on and model. And that takes a significant amount of time, just as an aside, if if you are looking at doing modeling using machine learning, your data scientist is going to be spending about 80 to 90 percent of the time just what we call data wrangling, just getting finding the relevant data and and making sure that it is in a machine readable format that is relevant for what your use case is, and is standardized in a way that is going to actually be processed and ingested uh as the input for an algorithm. Um, it's really, really hard to do that. It's very time consuming and costly. But we also are able to rely on some uh public data sets, most of them are coming from from the government and academia. The one that this particular project uses is a really massive, very interesting data set called Severe. So it's the storm event imagery data set that we're being that this particular project is using. And that data set we've been able to refine, we've been able to add to it, and we pushed that out with a public challenge. And so each of our projects, we have uh about 13, 12, 13, 14 projects at any given time. And this is one of those, but each of those projects is required to come up with a public challenge. And as part of the public challenge, it is publishing a data set, typically having some starter code uh that is uh produced. So basically it's it's copyright free code that that's able to be uh used to help help actually do everything from extracting the data and able to actually start processing it, as well as kind of a baseline algorithm to start modeling and a prompt, basically the problem that we're trying to solve. And so for this particular problem, we're using this this this severe data set. And what that is, it's a is a large curated data set that has radar imagery. So a lot of video and and images of of radar. Uh, and what you're what you're looking at is basically the sensors and it's it's like hour-long segments. The challenge is to be able to look at the hour-long segment of radar imagery and come up with an algorithmic way to predict what that next hour is going to look like. And so that's that's the challenge. And so we push out this really interesting data set of of weather radar imagery and and this challenge with the starter code, and we invite really the the rest of the world to to take a stab at solving that problem because it's very challenging.
SPEAKER_01So, Andy, as the national one of the national security lawyers in this discussion, my ears perked up when you talked about the ITAR, sort of the international traffic and arms regulation. And I thought it would be really interesting to kind of drill down a little bit and your role as chief legal counsel, sort of how your world intersects with weather forecasting, AI, algorithms, sort of what the universe of actually what you're doing with your legal hat on looks like in this space.
SPEAKER_00Sure. So that I'll say right at the bat, it's it's an incredible job. Um it definitely keeps me on my toes, and I feel very gamefully employed at the moment. And things like that. So that like that question on ITAR, when it comes to you know research that the Air Force is conducting on mission-relevant AI, uh, it sounds really scary to national security lawyers. But the this is part of a policy and really a national security strategy perspective. In fact, the national security strategy uh from this administration was just published uh last week. And in there, you'll see many times where it's talking about the role of public data and engaging not only just within the government, uh, but in a whole nation effort uh to solve these big problems. And some of those are specifically discussed here is climate. And so, how does the the government interact with academia, interact with industry in a way that, and as well as our our allied partners in a way to solve these really big societal, uh not only American but international problems? So, weather is a great example of that. Uh, each of our projects is is going to be by net definition a dual use case where there's a civilian as well as a military problem that's being solved. And so when it comes to how that decision is made, that's made at the outset, and we we were very you know careful throughout the process, but we certainly treat that as an asset that we're able to work with really the top minds in in their various fields, regardless of their nationality. And they're on their very carefully scoped projects to ensure that we we don't you know butt into any of those ITAR issues or or other issues, but it allows us to, as I mentioned, really tap into phenomenal talent that we wouldn't have access to otherwise. But it also provides us in the Air Force that are at the accelerator with some just unrivaled perspective when it comes to thinking through how do we apply AI to our problems? How do we do, how do we conduct research in this area? How do we think about what it means to be using AI responsibly? And we get the the whole panopoly of perspectives just in our own organization. Uh, and we have weekly meetings with the the principal investigators, the lab directors, the the postdocs, the the undergrad students, the the researchers, and the the airmen on the team. And we're able to discuss that the progress that we made on the research is very collegial. And we're able to talk about these really fascinating issues like ethics, uh, in a in a way that is is certainly enriching the people here. Uh I think that goes both ways. Um, that there's there's a lot of appreciation for the Air Force mission from the MIT faculty and students who probably wouldn't have had much exposure to that before. But between the the academic perspective, the international perspective, and the various communities that the airmen that are part of the accelerator come from. So I'm the lawyer. We out of the the 13 airmen that that we have permanently assigned to the accelerator, there's 10 different career fields. So we have intel officers, we have pilots, we have cyber operators, ops researchers, uh, as analysts. So really lots of different perspectives, lots of life experiences that are going to shade our opinions about the this type of research that we're doing, about AI generally, about the societal impact of AI. And we're able to actually have great friendly discussions that that advance our knowledge and appreciation of this across the board.
SPEAKER_05So I mean, I'm I'm a research person myself, and my wandering into AI seemed pretty natural for a computer scientist. As a lawyer, as an attorney, you know, someone trained in a completely different galaxy, what brought you to AI? Why? I mean, you're a PhD candidate now in with AI, and I'm wondering how did you get there?
SPEAKER_00So I've I've always been just drawn to how the military and really specifically the Air Force, for as long as I can remember, uses technology. And I it's just a just a fascination. Uh, and I've been incredibly lucky in my Air Force career. Uh, I've been in for a little over 12 years now. I've had some some great experiences in in operations, in in the courtroom, uh, but primarily in in government contracts. So uh I used to teach at the Army Jag School as a professor of of government contracts. And during that time, I was able to work on a basically while I was teaching, also consulting various agencies within the within the federal government, happened to start working on some some of the first AI projects in the DOD. And it really just kind of lined up with my personal interest and and able to you know allow me to, from my professional lens, this at the time very new technology. And you think about AI, it's we've heard about for years, and much of what we know is is really rooted in science fiction and in uh this Hollywood sensationalism, but the state of the art as we know it, the emergence from the so-called AI winter really happened in 2012. And so that's after I started my Air Force career, and and that's when machine learning really became the state of the art and such a powerful tool because we have this convergence of really the three pillars of machine learning. You have very, very copious data coming from sensors all over the place. You think about the internet of things, uh, that's providing structured data that's that's very easy to drive insight from, and it's uh been able to show and improve its worth in the marketplace, and starting to do so in the battle space now as well. You have incredibly fast and efficient and now fairly affordable, well, certainly relatively affordable computing power, and you have very optimized algorithms, and so all three of those come together to create the state that we're in right now where AI is so powerful. But just a few years ago, it certainly wasn't. And when I was able to work on really some of the first uh AI programs in the DOD, uh that's what really focused me from just being generally appreciative and curious about a technology to being just completely all in and focused on on a particular technology because it's just AI is just an interesting, it's not a technology, it's a it's a ubiquitous technology enabler. It affects all other technologies. And so it and it has just impacts on on all facets of society, private sector, public sector, and from a from a legal perspective, it doesn't matter what type of lawyer you are, you're going to touch AI. You're going to have to roll up your sleeves and get your hands dirty and understand how this technology works, because there's legal questions and it's really pushing the boundaries of what the law covers. We're seeing this across the world. Uh, the EU has been probably, in my opinion, leading in terms of trying to get ahead and understand and be proactive instead of reactive to the technology. And now they the EU is much more of a has a regulatory mindset than the US, which is much more of an open marketplace system. And so what they're doing might not translate particularly well to the US system, but it is really interesting to see. It's you know, it is going to affect criminal law. It already has. It affects privacy, so First Amendment issues, Fourth Amendment issues. It contracts very clearly are implicated. One in terms of AI for contracts, and contracts for AI are going to be more complex and require a different perspective and understanding than would be required for other types of supplies and services.
SPEAKER_03Well, and and as the lawyers in our audience know, lawyers have an ethical duty to keep abreast of new technologies.
SPEAKER_00Right. We have a duty a competence to be able to advise and uh consult our clients and provide recommendations. And if you if you don't understand, you know, what exactly what how is this data interacting? Who's you know, where's this data coming from? Where is it going? Who gets to see it? That's that all those are all questions that are going to have uh important legal uh factors that need to be considered.
SPEAKER_03As you were talking about weather, I was thinking how important the weather was to the Wright brothers and the fact that they chose Kitty Hawk in part because they wanted to have a certain kind of weather, but it varied from day to day as to what they were going to be able to do. But the developed world is covered with a rich web of weather sensors, as we know. But what about the developing world's weather, which I'm I'm guessing is much less well measured. And it's certainly as climate change accelerates, parts of the developing world are getting stressed in the extremes, uh, extreme drought, extreme storms, storms that are predicted to be of one severity, and then several days later, as happened even in the developed world down in Sydney. I don't think they knew what they were in for when the floods hit recently. Uh, how does your research, do you think, when I say yours, I mean that going on at the AI accelerator, hope to improve weather situational awareness around the world?
SPEAKER_00So, Roland, that's that's a huge problem, and your your inclination is is spot on. The developing world is is much less represented by the types of sensors that were typically relied upon uh for weather forecasting. And so, how how do you address that? And what one way is is as I mentioned before is that that global synthetic radar that that's able to make sense of various sensor data from uh multimodalities uh and and stitch it together to help create, you know, it essentially appears to be a weather radar. It's it's synthetic, but it it is pretty pretty accurate and certainly better than better than nothing. Now, from an Air Force perspective, that's incredibly important, you know, for the the very basic reason of we operate globally. We don't just operate in the developed world. And so we we have our mission in those areas, and so we we do need to understand what that that weather is because it affects the mission. From a broader picture, uh as you mentioned, it's weather uh and and weather patterns and weather events is certainly a national security risk, but it's really an international security risk. Um, and as you start to see these patterns develop of uh you know of extreme weather events that are only increasing, that cause diasporas, it causes uh you know climate refugees, it causes potentially famines, uh extinction events of you know critical wildlife and deforestation fires, it it causes obviously many issues. And the Air Force, uh, the more probably the most recent example is Tyndall Air Force Base, which is a pretty critical training base down in Panhandle, Florida, was all but destroyed a few years ago by a hurricane subsequent flooding. And so a lot of the climate action plan, and the Air Force just released its climate action plan a few weeks ago. One of the areas that it discusses is building resilient bases. And that certainly means within the United States, but also means really worldwide, from an Air Force perspective, it's it's really to ensure that it's able to conduct its mission regardless of the weather, or at least um do the best it's can to uh you know in even extreme weather. But the types of uh research and the resulting technology and courses of action the Air Force is going to take to be able to meet that objective are going to have um play a role in in the other types of research, whether it's material research on what types of like concrete or um or or building materials are going to be able to withstand the types of uh pressures that that are brought on by uh extreme weather events, that that's all uh going to have an impact outside the Air Force as well.
SPEAKER_03So these defense developments that are producing benefits for the Air Force, um, do you think there are specific immediate spin-offs or benefits for civilian activities, whether it's forecasting, weather for farmers or some of these large-scale uh early warnings? Because I know for many of us, if we just go back 10 or 15 years, we didn't have the kind of warnings that we do now about when to start preparing for a hurricane. And and we shouldn't overlook just how much progress has been made there. But I've never heard anybody say that progress was from AI. So I guess what I'm asking you is can you talk about what specific civilian benefits you think will come from these AI weather projects that are going on at MIT?
SPEAKER_00So from the the very pure weather problem perspective, AI is is really good at detecting anomalies. And if you look at the way that the weather has been predicted before, it's relying on uh historic data. And you know, there might not have been that many anomalies in the in the past. And so is the historic data a good, relevant data set to use to predict? I think that's a that's a an interesting uh you know earth science question that that is is being addressed. And so looking at that is okay, how can we how can we also predict things that didn't happen before, unprecedented events? Well, one thing that that machine learning is is very, very good at is detecting anomalies. So identifying where the outliers are at, and and if it's if the the data, the signal data that is coming in is you know, there's it, you know, the algorithm's able to to identify this is an outlier. Usually what it is able to do is able to basically label and in and bring it to an analyst to to actually review it and make sure, yes, this is an outlier. Sometimes you know it's it's different, but you don't know what that means. And sometimes it's different enough, like we should look into this more. And that has various applications, uh, but certainly in weather as well. So from the weather perspective, I think outlier detection is going to be really important in in helping that that particular issue. What do how do we get that early warning? And so that that to me seems like a really powerful way that AI can help. Uh, and that that goes to civilian as well as the Air Force. There's also a lot of ways where our severe weather, uh our severe data set that we're using, it's such a really interesting data set. And weather is such a a classical uh statistical model. You think of, you know, what does machine learning do? Well, it's a it's a prediction, it's a statistical model. Well, what else do we know that we do that in and that's it's weather? And so by by using this classical model and this really large data set, we're able to do run some other experiments that are are going to really advance the state of the art of machine learning science just across the board. And so a couple of examples of that are explainability. One of the problems with with advanced machine learning techniques like deep neural networks is these algorithms are are there's so many different parameters, or so, and because yet the the algorithms are being constantly rewritten by the data that it's ingesting, you have this black box of a model where even the data scientists that create this algorithm are not necessarily going to be able to explain why it's making these types of predictions. And so a very large data set is used as a proxy to help understand ways to improve explainability for neural networks. And so that is going to have a profound impact on the way that we're able to, and and really on the judgment call of whether we even use machine learning for a particular application. If it's higher risk, if it's not, if we can't explain it, well, maybe we don't want to use it. There's a lot of there's a lot of types of cases in the in certainly in the Air Force and defense context, where maybe we just want to know what's going on for various reasons. Well, having an explainable model is really going to be helpful. Another way that this particular data set, uh in just the context of of this weather problem is being used is a way called few shot learning. So few shot learnings. So instead of having massive amounts of data to train a model, which is typically required, the really the the old adage is that the more data, the more representative, relevant data that you have, the the more accurate your model is going to be. Well, sometimes, oftentimes, I should say, we just don't have all that data. We have what's what most would consider too small of a data set. Well, the concept, the idea behind the few shot learning project, uh, again, using this this severe weather data set, is maybe if we are able to get really, really good examples that are very representative, very relevant to a very specific use case, say like 20, 25 great examples, that can actually result in an accurate model. Now, there's going to be other trade-offs with that, but it's it does kind of go against the grain in the traditional way of thinking about machine learning and in just deep learning in general, that we need to have lots of data. Now, that's good for practical reasons. If you can't get a hold of all the all the data, you still want to be able to derive some insight in it and make some sort of prediction. And you also want to know about how accurate that prediction is going to be based on the limited data. But by doing that, you're actually saving a tremendous amount of cost and energy. One in just coming up with all the data, uh, but two, in the actual computation and and the training of that model, deep neural networks are notoriously power hungry, like extremely so. And so few shot learning is is able to address that. So we have AI to help inform us about weather, and we also have AI to help mitigate some of the weather, uh, you know, to the extent that there are contributions of uh to the extreme weather patterns from industrial use cases and you know, in and and machine learning specifically, ways to mitigate that. And so that's that's also, of course, going to have an impact beyond just the Air Force.
SPEAKER_05This is again, I keep saying this is fascinating, but uh it is. Many of the things you're talking about, your potential benefits, and maybe this, maybe that. I don't want to sound like a funding agency, but how soon do you expect to see these kinds of things start trickling out and actually impacting daily lives?
SPEAKER_00So, based on the type of research that we're doing, uh it is fundamental research. And so that that's it is on the earlier stage of the technical readiness cycle. And really the reason why we're going to MIT is and and Lincoln Lab is they're they're very fond of tackling the world's hardest problems. And so the Air Force did its best job to bring the world's hardest problems. And and we kind of joke around that you know, what we do at the accelerators really make the impossible very, very difficult. And so some of the things that we're doing were thought to, you know, they they are true discoveries. So we we've had a few patents that have been filed, we have pension disclosures. We've also had over 200 publications um that have pushed out our our research to to the public. Our public challenges, as I mentioned, are ways for us to take really interesting data sets, very hard and relevant problems, and invite the rest of the world to work on that. And we provide the not only the data, but also the software, um, the starter code on our GitHub. And so that's open to the public. So we are working on really that early stage, and it's going to take other other organizations, other universities and companies and individuals and governments to continue that research that we're doing and advance it. So, you know, I'd like nothing more than to see the uh you know, the the publications that the AI accelerator has has pushed out in the past couple of years. We've we've only been around since May of 2019 is when we're founded. We really didn't get funding for a few months later. And so the type of research that the the team has has been able to do uh is pretty great, considering, you know, so far I really only talked about one of the like 12 or 13 projects. So we do have some that are farther, you know, from a technical readiness level. Um, you know, most of most of our projects start at like a one or two level. We have a few that are coming up to like the four or five level where we're actually doing some some testing uh in operational environments.
SPEAKER_03I don't think we understand the numbering schemes.
SPEAKER_00Sure, yeah. So so the technical readiness level is just a it's a shorthand way of describing the where the technical maturity of a particular research project's at. And so uh most startups operate in like the TRL, uh like six seven world. Um any university research is going to be in the one through four range. That's kind of where you see like DARPA projects are going to be at at that level, programs um of record for the DOD. So basically our our systems acquisitions are gonna be kind of in that that six seven uh level as well. And the things that you just go out and buy are are gonna be in a in a TRL nine. That's kind of the end state where it's a proven concept, it works, it's it's repeatable, it's it's basically you know ready to go. And so it's there's a it's a process for that and aligns fairly well with our acquisition process. But you know, the the point of that is a lot of this is is very early, and there there needs to be, there's still a lot of work that remains uh in in this area. It's no secret that this is very challenging work um that our team is doing.
SPEAKER_01So, Andy, as as these projects that you're mentioning sort of progress and as these true discoveries sort of come to fruition, I'm curious, can you give us a little bit of a snapshot into some of the ethical issues that you and your teams have been grappling with or seeing down the line as this technology emerges?
SPEAKER_00Yeah, we we spend a lot of time talking about the the ethical issues and kind of from the our the AI accelerator, so my relationship with MIT, and then um the you know we we're plugged in and part of the the ecosystem within the Department of the Defense and their chief uh digital and artificial intelligence office as well. That's kind of the successor of the Jake, the Joint Artificial Intelligence Center, and they're the ones that are really working for the direction of the Deputy Secretary of Defense uh and Secretary Hicks on responsible AI. And so that that's that's essentially the term that we're using to talk about the the way that the Department of Defense is going to use AI. Uh, and so from a research perspective, we're looking at at many different ways to ensure that you know it's the types of use. That we're really advancing are going to be able to be used reliably and are going to essentially do what their intended purpose is. We want to be able to understand why it's doing it, what it's doing, uh, whether it's doing the intended purpose or the unintended purpose. Um there's unattended activity as well. We want to be able to understand what does it need from here's from the lawyer perspective, what types of data rights, what types of uh license into intellectual property from a commercial platform would the Air Force need to be able to ensure that it is it you know adhering to its responsible AI tenants. And so we're we're looking at it from a kind of a research perspective, a policy perspective, a legal perspective. And you know, certainly we we discussed the just the broader uh societal impact as well.
SPEAKER_03Andrew, you mentioned interestingly that you know lawyers who are familiar, for example, with software acquisition projects are going to need to learn AI because they won't be the same. I think a question that's sort of hovering in the background. We know that DOD projects, acquisition projects tend to take longer than the DOD or the taxpayer would like. They tend to cost more than anticipated. Is acquiring AI systems, are those gonna take even longer? Or do you think these we're gonna get the upper hand in this acquisition process? Because I think there is concern that our peer adversaries are finding ways to do this faster. And in some areas they're made strides ahead of us, and we have a catch-up to do. And if we're gonna take a long time to do it, we're not likely to catch up at all.
SPEAKER_00No, that's an excellent question. So I think this is probably an appropriate time to mention the the standard DOD disclaimer that uh you know what I say here is is really my opinions and uh in and may or or may not uh reflect the the the the positions of the department of defense or any of its organizations. Um and I say that as a as a preface of there's there's not really a written standard or or or uh you know DOD tagline on on what this means. And so this is this is purely my opinion. But to answer your question, Roll, and I I certainly hope we don't take longer. Um I I think time is of the essence for for a couple of reasons. One, as you said, to uh maintain our our uh competitive advantage uh such that it is in AI. We do need to move with expediency in the the National Security Commission on AI that came out uh in in March of last year, 2021, suggested that the the the United States is is not ready to defend and compete in the AI era, and it gave us a 750 some odd page get well plan with the target of of 2025 to do better. And much of that content is is really describing how do we buy AI. Uh, it's not the only thing, that's not the only problem um that the commission uh discussed, but it is certainly a big one. So acquisition in general has been studied and tinkered with for decades, certainly throughout my entire career and and much, much, much longer before I started working in the federal government, and a lot along the lines of how how do we get this right? Well, AI has really brought into the forefront how important it is to go fast. One, you know, we have some serious competition in a way that we haven't had before in the form of China that is committed to leading the world in AI, and they're doing a really good job at taking a lot of those quantitative boxes. And you know, we're we're still in in many qualitative metrics, we're still slow leading, but we want to we want to keep it that way. And and being able to acquire, not only acquire AI, but to there's a lot of work after you buy it to actually deploy it as well. And so, you know, just buying it just one step. And and so that from a contract perspective is unusual in that uh it requires a lot of coordination, requires a lot of user feedback. Sure, right. Well, maintenance or or continuous development, it's uh it's it's you know, they're it's hard to distinguish the two when we're talking about an AI system. And so where are you getting the data from? Is it you know, are we going to be continuously training it, or is it going to be a train pre-trained model that we're just going to evaluate? You know, is the data all is it all government-owned data? Is it uh proprietary data? Is it open source data? All answers to those questions have have serious acquisition impact. And from an acquisition strategy perspective, from a contract perspective, that's going to be important. And so, yeah, we should go faster, one for for just a pure national security reason, but we're also need to go faster because we're working with companies that are not the traditional companies that the Air Force is used to working with. So if you look at where AI innovation is coming from, you can see there are companies in Silicon Valley that are not part of the defense innovation group. It is they're they're the the big like Googles and Amazons and Facebooks of the world that they actually spend more on uh research and development for AI than the federal government spends on science and technology research. So there's this huge disparity, and they're not accustomed to going through the somewhat arduous, lengthy procurement cycles that the Department of Defense is known for. And so getting not to mention this technology moves fast. You go through multiple iterations in a standard procurement cycle, and so you don't want to be buying obsolete technology, you want to be buying cutting-edge technology. So speed is key. So we have these companies that are working in the US, and so how do we attract them? How do we align our contract practices to be able to one account for the technological concerns, but also the business concerns uh is is going to be a challenge for for us. And I'm happy to be part of uh you know the group looking into how do we do how do we get this right? And certainly that's that's basically what I described is the topic of my PhD thesis. So I've been thinking about this for a while now. And I do think that contract law is a huge factor in the in the the overall equation of how how do we bring AI into uh into the Air Force.
SPEAKER_03Well, Major Baum, this has been a wonderful conversation. We very much appreciate your taking the time and walking the fine line between what you could tell us and what you couldn't. We hope to have you back sometime because I think you mentioned that another area of your research is in navigation without relying on GPS satellites. And I think we'd be keen to hear about that if you can make time for us. But in the meantime, thank you so much. On behalf of all of us, thanks for being on this podcast with us.
SPEAKER_00Well, thank you. This has been my absolute pleasure.
SPEAKER_05We thank the business law section of the American Bar Association for their generous sponsorship of the production of this podcast. We welcome questions and comments from listeners. Send email to comments at mind of the gap dialogues.com. We read all comments and questions and will try to respond in the letters section of a future episode. If you're writing about a particular episode, please mention the specific episode number. And please also include pronunciation tips to help us properly say your name when we reply in a subsequent episode. See you next time on Mind the Gap Dialogues on AI.
SPEAKER_02Thank you for listening to the ABA Business Law Section podcast series to the extent that the section offers a robust collection of content. To explore more about this topic or to learn about joining the section, visit ambar.org slash bizlaw. That's B-I-D-L-A-W.