Living With AI Podcast: Challenges of Living with Artificial Intelligence

This podcast digs into key issues that arise when building, operating, and using machines and apps that are powered by artificial intelligence. We look at industry, homes and cities. AI is increasingly being used to help optimise our lives, making software and machines faster, more precise, and generally easier to use. However, they also raise concerns when they fail, misuse our data, or are too complex for the users to understand their implications. Set up by the UKRI Trustworthy Autonomous Systems Hub this podcast brings in experts in the field from Industry & Academia to discuss Robots in Space, Driverless Cars, Autonomous Ships, Drones, Covid-19 Track & Trace and much more. If you have any questions or suggestions for future topics, please email us at contact@tas.ac.uk - You can find out more on our website at www.tas.ac.uk#LWAIP #TASHub

All Episodes

Living With AI Podcast: Challenges of Living with Artificial Intelligence

Trusting Autonomous Ships

July 06, 2022 • Sean Riley • Season 2 • Episode 4

Featuring Manuel & Pedro from Ocean Infinity, we're discussing the role of autonomous water craft - how can you regulate them? What problems do they solve? How can we trust them to be safe?

Featured Interview:
Manuel Parente: Software Technology Development at Ocean Infinity
Pedro Costa: AI Lead at Ocean Infinity

Panel Guests:
Katrina Kemp - Maritime & Coastguard Agency
Jon Downes - Associate Professor in Maritime & Autonomous Systems, University of Southampton

Podcast production by boardie.com

Podcast Host: Sean Riley

Producer: Louise Male

If you want to get in touch with us here at the Living with AI Podcast, you can visit the TAS Hub website at www.tas.ac.uk where you can also find out more about the Trustworthy Autonomous Systems Hub Living With AI Podcast.

Podcast Host: Sean Riley

The UKRI Trustworthy Autonomous Systems (TAS) Hub Website

Challenges of Living with Artificial Intelligence

Season: 2, Episode: 4

Trusting Autonomous Ships

Episode Transcript:

Sean: You’re listening to Living With AI, season two episode four. This is the podcast where we discuss how artificial intelligence is changing our lives. This week we’re talking about autonomous shipping. I’m Sean Riley, you’re host here on Living With Ai. Now shortly we’ll hear from Manuel and Pedro from Ocean Infinity but before that let’s meet this week’s panel. Today we have Katrina and Jon. I’ll ask you to introduce yourselves so just let us know a little bit about what you do. So Katrina.

Katrina: Hi, I’m Katrina Kemp. I work for the Maritime and Coastguard Agency. My job is essentially to start making the regulations for autonomous ships both in the UK but also internationally at the International Maritime Organisation.

Jon: Hi, I’m Jon Downes. I’m an associate professor in maritime autonomous systems at the University of Southampton working with a range of surface and subsurface craft and working with the integration of those platforms into wider systems.

Sean: Thanks to both of you for being here. Well we’ll have a listen to the feature in a moment but before we do, just to let our listeners know we’re recording this on 21st June 2022 so hopefully that gives you a little bit of context in case you’re sitting in an autonomous hovercraft backed up in traffic near the Suez Canal or something equally futuristic. I’d like to introduce Manuel and Pedro from Ocean Infinity now. Great to have you on the podcast. Can you just tell us a little bit, what’s your name, what do you do, what does your organisation do. So Manuel, shall we start with you?

Manuel: Thank you so much for the invitation. I’m a very big fan of you. So my name is Manuel Parente. I oversee the technology development specifically around software for the Ocean Infinity company. We have several development groups around the world. Right now the biggest challenge that we’re facing is how can we become fully autonomous. So essentially it’s creating a fully autonomous vessel and fully autonomous operations in the near future. That’s the big challenge that we’re tackling right now.

But essentially, more importantly, it’s how can we do it in a sustainable way. So before joining Ocean Infinity, I was the CTO and founder of Abyssal, a company just focused on the development of augmented reality and artificial intelligence for submarines, essentially remotely controlled vehicles.

Pedro: Thank you for the invitation. My name is Pedro Costa. I am basically the AI guy here so I am the lead of artificial intelligence at Ocean Infinity so my responsibility is to plan and develop projects on perception, planning and control systems for vessels and underwater vehicles as well. On top of that I’m also a PhD student here at the University of Porto where I’m trying to develop some unsupervised segmentation models, so basically some models that can identify things that we see in images and videos without any annotated data.

Sean: The one thing that I think most people, certainly I think of when I think of autonomous shipping is the connection to SpaceX and we see this ship sail in and a rocket land on it and it all looks absolutely fantastic. Is that where we’re at with things now? What’s the state of the art with autonomous shipping right now?

Manuel: There is a parallel on the difficulty it is to set up a base on Mars or a base on the moon and the difficulty that it is to navigate and do work in a social environment because if you look at the social environment working at 3,000 metres of depth. In terms of pressure it will be around 300 times the pressure of the pressure inside a tyre so it’s quite difficult to work in that environment. Plus you have zero visibility many times, you have no natural light so all of this compounds to a very complex environment.

Now the challenge here is not only we have to create an environment where we can safely and sustainably navigate a very big vessel, we also have to stop the vessel, launch an [unclear 00:04:28] which is basically the size of a car, navigate down to 3,000 metres in depth and then execute manipulator functions. Then another difficulty, and this is where the zeitgeist changes and it creates a difficult challenge, we have low bandwidth, we have to use satellite communications. So we’ll have low bandwidth and high latency.

That means we have to create a system in which we cannot synchronously control a robot, so we cannot use a joystick and control that will be live. We have to create the environment where we send the commands and that robot has some sort of autonomy or understanding of what that command means and execute that command because essentially you have no guarantee of communication. At any time you can lose that communication just because you have clouds in the sky and many other situations where you lose communication. So what is the state of the art? Well I wouldn’t say that any of them is easier than the other. I would say that what we’re trying to tackle is as difficult as setting up a base on the moon.

Sean: What sorts of tasks would submersible need to do that you would be controlling in this way or activating in this way? What sorts of things are you doing? Is this like cable laying and things like this?

Manuel: Well if you look at just the vessel itself, we have operations in which just the vessel itself can perform, survey operations but just a simple, I shouldn’t call it simple because it’s not simple at all but in comparison it is. Just the fact of doing a survey, you have maybe 20 or 30 different sensors and equipment that are gathering data at maybe 10 hertz and we’re generating so much data and we have to process so much data locally on the vessel that even the process data is bigger than the amount of bandwidth we have to send it home.

So we still have to create a lot of, I would say autonomous functions, autonomous behaviour just for a simple, I would say a simple task but it’s highly complex. We then have, let me see, the vessel could be gathering geographical data. It could have a geo tech drill. It’s equipment that we use just to sample the social environment. There are crane operations. There are so many other operations. That’s only on the vessel. When you go into social environment, we then have AVs and AVs do pipeline inspections or do-

Sean: So checking on things, it’s presumably quite dangerous to send a human to go down there and do.

Manuel: Then you go on the realm of ROVs and those are remotely operated vehicles and they have manipulator arms and they do all sorts of things like for example in oil and gas fields they do all sorts of inspection, maintenance, repair, cutting, there’s so many complex aspects of the operations that these robots have to be capable of doing on their own.

Sean: So it is comparable to operating a Mars Rover or something like this?

Manuel: Yes.

Sean: Coming over to Pedro briefly then. You mentioned AI is your specialty so how do you get enough data? I mean in my understanding, AI needs usually a lot of data in to be able to be trained to work out how to go about doing its task. Where has that data come from or are you looking at doing it a different way?

Pedro: I think you touched on the critical point here. That’s a really, really big issue that I think we’re facing, not we as Ocean Infinity but we as an industry and also one of the reasons why for instance there’s not a lot of universities and research groups working on this type of industry because well, when we talk about self-driving cars, anyone has a car or most people have a car. It’s very easy to just put a camera on top of the car and just collect data.

If we want to collect data let’s say to train perception and planning algorithms or to train artificial intelligence models for a big vessel, it’s not that simple. We need to just go with the vessel in the middle of the ocean or the sea and we need a big crew and that’s very expensive. So that’s why there is no publicly available data to develop autonomous ships or autonomous vessels. Whereas there is a lot of publicly available data for autonomous or self-driving cars. So that’s actually something critical.

One of the things we’re doing right now is we’re equipping all our ships with as much sensors as we can and try to collect as much data as we can on our ongoing operations. So we try to store all that data but still, it’s challenging because, well it’s not the same as in the surface environment where you can just launch a car and drive it somewhere else and collect the data that you want.

Sean: In very simplistic terms, it seems to me that you could naïvely say well the ocean’s a massive place most of the time GPS works, send the ship out. I man I understand that weather is going to get in the way but what else has it got to work out apart from docking perhaps and a bit of obstacle avoidance?

Manuel: Weather is a big part of it but navigating safely and building trust is also a factor. We don’t have enough data yet to safely say that we have an autonomous function of a certain level but we are building up to it. So we understand the vessels that we’re building have to be very complete in terms of all data. From the software perspective, we want to capture any and all data that the vessels are producing, including the inputs from the crew so that we have enough data to then build the autonomacy.

The main pillar of what Ocean Infinity is trying to do is how to do all of these operations sustainability and with the least amount of Co2 emissions possible. An example of how [unclear 00:11:39] could help us is if we are doing route planning and we can predict weather in certain areas. We can re-route that vessel for the least amount of effort and the least amount of effort. The least amount of effort equates to the least amount of engine and that equates to the least amount of Co2.

So I think we will gain a lot with compiling all of this different data and use it for our benefit and that data is third party weather service forecasts and other areas like for example there are other challenges like with data so we are using low earth orbit satellite communication and that means that knowing where they are and what the coverage is and forecasting that coverage along the route can also help us predict when is the best possible geographical location to upload the data that we are capturing. For us it’s just like [unclear 00:12:40]. For us a vessel is just a very big floating server rack.

Sean: A lot of senses, a lot of data. You mentioned levels there, that’s quite interesting because I think many people have heard of the different levels of autonomy in self-driving cars. Could one of you perhaps take us through how that works with shipping. Is it similar? Is there a different scale? How does that work?

Pedro: I think we can use exactly the same levels of autonomy as the ones that are used for self-driving cars. I think those levels are useful to try to understand what tasks we should automate depending on the risk associated with a certain task. I think that also ties in with the trust in the actual system. So we much more easily trust a system when there is very few risk involved if that system fails other than well in situations it’s much harder to trust a system where the potential outcome is people dying or just having an environmental disaster, very bad thing happen.

So one of the things we’re trying to do and that’s where self-driving cars are also approaching which is supervised autonomy. So right now most of the cars that are driving on the street, they must have a pilot controlling them or supervising how the system is performing. One thing we’re trying to do is we’re trying to, and this is like a conflicting goal with this sort of supervising autonomy, we’re trying to remove people from the vessels. So we’re trying to remove people from the vessels because it’s safer for people and also we can develop vessels that are more environmentally friendly. If we move people from the ships, we can reduce the size of the ships and then emit less carbon dioxide.

So we’re starting to develop remote control capabilities. So that’s where we want to move in the future. So we want to have the possibility of having supervised autonomy in the vessels but before we can do that, we’ll need to be able to have remote control vessels. So there is a pilot [unclear 00:15:41] and supervising the autonomous system basically.

Sean: Most of these autonomous systems in general are only probably as good as their sensors. I mean are we talking about equipping these ships with a thousand cameras? What sort of information will they need to go about doing what they need to do?

Pedro: So we’ll need many, many cameras, that’s one thing. We’ll need radar, obviously GPS. So those are, I think, the main sensors we need. We need to then fuse information coming from radar with information coming from cameras but we have another additional challenge here which is we have vessels from different dimensions. We have 78 metre vessels. We’re starting to build 83.

Manuel: 85.

Pedro: 85 metre vessel as well and we have smaller vessels. So the sensor stack that we include in these different vessels will also depend on the size of the vessel but yes, we definitely need to have 360 degree coverage of the vessel like in self-driving cars. You can imagine on an 85 metre vessel, that’s going to take a lot of cameras. Again as Manuel mentioned, that’s a lot of bandwidth. We cannot just feed all the video coming from all those cameras in real time to shore.

Not only we will need to have these 360 around the vessel, we’ll also need to have cameras inside the vessel so we can spot if there’s something failing, for instance if there is an oil leakage coming from one of the vessel motors for instance, so we can identify if something is failing. So we’ll also need those sensors inside the vessel as well which, well many, many, many different sensors.

Sean: That’s an interesting point because these ships generally travel with crews of engineers and all sorts, don’t they, to repair these. What’s the way to mitigate if there are problems? Is that then you have to, I don’t know, drop anchor and send somebody out to fix it? What happens?

Manuel: That’s one of the possibilities. We have different levels of danger or criticality. The last level is dropping anchor and wait for another of our own vessels to go there and fix it but that’s the last resort. We also have situations where maybe the vessel fails and we don’t have, not even dropping anchor is possible so we also have contingency plans for that. But in essence you have to rethink everything when there is no people on board. So we’re sustainably trying to build to that point.

One thing that Pedro mentioned was the middle step of remotely control the vessel and then fully autonomous but we have to think of them in parallel because again, you don’t have the guarantee of communication. As soon as you have bad weather, you will have intermittent communication. So you already have a situation where you have to train the vessel to react in those situations but we have contingencies like all over the globe we have suppliers that have a crew and a ship on call if something happens but that is still a long way. I mean four years’ time.

For now we have a leaned crew, so we have about 12 people inside the vessel that will take over if we lose communication. That’s the step to build trust. To build trust we have to calculate the amount of time that we have local human input versus the time we have remote and we have to learn from that experience. For that you talked about the policy and regulations, we have the IMO which also proposed four levels of mass autonomy, so autonomy for these vessels. That’s on an international level. For a national level we’re working with coastguards. There’s an IACS which is the classification society that are providing the technical input. All of these entities are part of the conversation.

Sean: People often expect a higher performance for automated systems than they do for human operated systems which sometimes seems a little unfair because you should at least be about the same perhaps. Is that something that you’re finding as well, that people are expecting them to be better than a human crew vehicle?

Pedro: Yes, I think so and that’s one thing I mentioned before, that it’s very hard to create trust in a system and very easy to just destroy that trust. We can just be building trust for two years, and what I mean by this is just having an autonomous system without any issue, without any incident and then one incident can just ruin all that trust that we built. So that’s something we are aware and I think regulators are aware as well. So we as developers need to be the ones that distrust the system the most.

So we need to make sure that we have all the quality assurance processes in place, that we just put the system under the most difficult scenarios, that we have all the test cases in place so we try to avoid that happening. But again, it’s a matter of time. So for instance last month there was, just to give you an example again the parallel self-driving cars, so last month in China the government allowed Pony AI and Baidu to operate robot access without a human driver in Beijing. It’s only on a small area.

For that, these companies had to be over 24 months of testing, over one million kilometres of testing as well and over 200,000 kilometres in Beijing to be able to achieve that. So that’s something that we built over time by placing these autonomous systems working in parallel with humans and just give them enough time so regulators and ourselves trust that the system can operate at least as well as a human being.

Sean: We’ve talked about some very specialist tasks like ROVs or whatever or submersivals going under water and doing inspections and tests but the biggest area of shipping has got to be cargo and containerships. Last year we obviously saw this blockage of the Suez Canal by the Ever Given, what was it, six days or something and the entire world trade is disrupted for however long. Is autonomy going to play a part in these massive containerships at some point?

Manuel: It most definitely will. You say that you already have some technology being developed and some autonomacy on those container vessels. This is my opinion, my opinion alone, I think that we may be able to scale down the container and logistics environment. If we can create a vessel that has lower emissions, lower fuel consumption we can do more skill instead of a bigger vessel. That will also alleviate the supply chain because instead of going to a port where you have a massive container for a week just offloading all the containers you have, multiple vessels coming in at the same rate in which that port can accommodate.

Sean: That seems to make a lot of sense. I know this is from science fiction but there are often authors writing about things like future sailing boats, AI powered autonomous boats that are using the wind. You’ve mentioned sustainability a few times already. Are these areas that you know about that are being investigated, the idea of AI controlled sailing boats?

Manuel: I wish I knew more about this but I can give you an example of something that we’re actually researching on which is ammonia separation into fuel cells, hydrogenic fuel cells. We’re working actively on that with UK universities. We are working on different ways of powering those vessels. We already know that an example is, it’s kind of a wild idea but anyway, let’s say that you have a wind farm and that wind farm you have a series of fuel cells and you can use the wind power which is a hundred percent clean to power those batteries.

The electric vessel will come in, plug in, power and continue the journey. That will be a hundred percent clan energy. Those sort of things aren’t always on our mind because out objective is to be a neutral identifier and actually be a corporation and then carbon neutral by 2040 I think. So it’s a big challenge but there are many areas that we are actively researching on, be them crazy or not, we have to try them.

Sean: What would you say are the things that worry people most? We’ve talked about this idea of trust and having to run things in parallel or certainly being supervised. What worries people about automated or autonomous ships?

Manuel: I think it’s the change, it’s just the change. There are many people that look at this as I am losing my job or I work offshore and now I will work on shore and we’re reducing the workforce. I have a really interesting story but before that, our objective, the biggest difficulty in offshore work is to have a work-life balance. You spend lots of time onshore and it’s difficult when you enter a certain age and you want to build a family, it’s very difficult. So I think this has a positive impact because all of those people that work offshore can now work onshore and have a normal day to day job.

The other point is the offshore safety case. People are just sager. It’s one of our biggest [unclear 00:27:46]. But there’s an interesting story. When we were in the space age before computers, we have people that do the computations, they do the math calculations and they were called computers. When the electronic computer came along, there was also the sense of now these people which are by the way all women and they are all very smart women doing the mathematical calculations that took us to the moon.

When those computers came along, they didn’t think they would lose their jobs, they became software developers. They became the first programmers. So computers became programmers. It’s just a story I like to tell. So change is not losing a job, change is just change.

Sean: It’s just moving that workforce around, right? So yes, you become a robot manager rather than a worker or whatever. What does the future look like? What should we expect to see in say five to 10 years’ time?

Pedro: Well I think we should have remotely operated vessels. In order to have those remotely operated vessels, we’ll need to be able to mostly automate all operations under water, also sea operations or at least have the possibility of supervised autonomy but not that supervised, when I say it’s more like let’s say we want to use a manipulator of an ROV so [unclear 00:29:29] a robotic arm. Because of latency issues, we need to be able to just allow a pilot to just interact, turn that valve around and the robot needs to do that, perform that operation completely autonomously.

So in five years’ time, I feel like we need to automate all social operations and be able to remotely control vessels while at the same time, and as Manuel mentioned, having these autonomous systems in parallel with human pilots. So we’re just building the trust case, just getting data, understanding how the autonomous systems, what’s he commands, the actions that autonomous system outputs and what’s the commands and actions that a human pilot performs compared to so we can just built the trust, go to the regulators and try to get these autonomous systems approved but again with trust and safety in mind.

Sean: How do you develop these autonomous agents?

Manuel: It’s something interestingly that we have built before for the subsea vehicles but we are applying that to the surface vessels. So we built 3D world in which you can simulate surface conditions and simulate the behaviour of bodies in water and under water. That means that we- because this is about building trust around AI, with that simulation we have the entire world model and we can place our vessel in any place of the world, in any weather condition or in any situation.

We can mimic loss of communication, we can mimic lots of incredibly bad weather but also simulate things like the amount of effort that the vessel has to go from one place to another or how can we control multiple vessels at one like in a hive situation. But interestingly the simulation has a specific use in AI. If you create a simulator that is realistic, you can use the output, it’s called a synthetic data. You can use that output as means to train an AI model because to train an AI model, it’s not jut about data. You have to annotate the data.

The data from a simulator is automatically perfectly annotated. So that speeds up the process of training AI and training autonomy and placing the vessel in all the corner cases that we probably never will be but we have to make sure that the system can handle that situation.

Sean: It sounds like there’s a really good spin-off of some kind of ship simulator game here as well you need to investigate.

Manuel I guess so.

Sean: Manuel and Pedro, thank you so much for joining us today on Living With AI.

Pedro: Thank you.

Manuel: Thank you very much.

Sean: Wonderful to get an insight there on the state of the art of in the industry surrounding autonomous shipping. Well over to our panel now. Just a reminder we’ve got Katrina Kemp and Jon Downes here. A lot of ground covered there. Obviously industry has got certain commercial pressures dictating the direction it takes. I feel we ought to have the academia or the academics view on this so Jon, I’m going to throw over to you you’re your professors hat on.

Jon: I think I was a very interesting discussion covering a number of the key points and the key areas to be thought about and in particular leading into the obvious question of how do we trust both the systems but the wider question of how do we trust the data that’s coming from those systems in order to make the decisions and utilise the reason we’re putting the systems out there.

Sean: Thinking about trusting that data, I mean there’s one thing that came up again and again and this is a Katrina question I think, and that is the regulation behind all of this. I mean whatever data the devices are sending us, they’ve got to be able to be trusted and be safe in the water, haven’t they?

Katrina: Yes, completely. Part of our dilemma as a regulator is that it’s not a surprise that when the regulations were written, they were written with the intention there will be somebody on board. They’re not going to be there. So for us, we actually almost need to go back to basics with some of the regulations to make sure we have the powers to create these standards and requirements for autonomous ships. By creating those, it almost gives people confidence that actually these are safe, these can be trusted because there’s this system in place to approve them.

We’re not there yet but we’re on that journey to start that and it will take time. Part of that conversation will be data and that’s a really interesting one because it presents a new dilemma for us in the maritime industry. It’s where that data is held, who has access to that data, what standards should it be kept to, as well as the communication of that data between the ship, the operator and then us as a regulator and what we need to see to do our job. So it’s fascinating and there’s lots going on.

Sean: The thing with any regulation, I mean we talked about this once or twice in the podcast but it’s often reactive, isn’t it. It’s very difficult to foresee what direction these things are going. In fact it’s difficult to foresee, pardon the pun, any direction any autonomous vehicle is going to take, particularly with weather and things getting in the way. Obviously the sea is a treacherous place, right.

Katrina: You’re right. I think for us as a regulator, the biggest thing is trying to keep up with technology and we can’t do that. We have to be honest. We won’t ever keep up with technology. So our challenge is making sure that the regulations we do create are flexible enough to ensure that they can still be those innovations that will come in the future but still make sure that what is out there now is safe. So it’s an interesting one but yes, you’re right. Those challenges of the sea are no different to the challenges for the conventional shipping.

So we can address those but it’s trying to keep up from a regulatory perspective because it’s not just us finding the time to figure out what we want, it’s then getting that through, whether that’s the UK parliamentary system or whether it’s the international arena which is another kettle of fish. It’s a huge body of people to get it through and that takes time.

Sean: Just thinking, I know, Jon, you’ve got experience in present day of dealing with autonomous craft, haven’t you, is it a level three craft that you have some experience with? What’s the situation as it stands now, if you wanted to put to sea as it were?

Jon: Well levels, yes, that’s an interesting discussion to begin with as to which variety of levels an whose levels we’re using.

Sean: I did think that was the same because as part of my research I looked up the levels of autonomous shipping and then was surprised to hear it discussed in different ways. Is that a running thing in the industry or is that not just unique to this conversation I had with Manuel and Pedro then?

Jon: No. It’s an ongoing one. I think Katrina may be better placed to say where we’re going with it but yes, there are differing versions of what we mean by levels. In terms of the vessel we have here at Southampton, it’s a commercial grade vessel, probably best put as it can operate under remote supervision. So it’s always operated under a visual line of sight within about a kilometre within communication range of us. That’s both related to its size, its capabilities but also ensuring that we are trying to operate within best practice.

Sean: I mean I spoke to some people at a robotics show a few months ago, was it [unclear 00:38:13], these guys, and they had been sending research vessels quite a long way away from what I understood, right up into the Arctic circle. So is there some loophole? Actually Katrina, is there a loophole that these things are operating under at the moment? Is that going to be tightened up?

Katrina: That’s a mean question but no, there’s no loophole. As much as those regulations were written with ensuring there’s a person on board, there is always, because there are good lawyers behind this, the ability to provide exemptions. So we do have an exemption process and that’s how some of these vessels are operating. The difficulty is is there’s no international regulation. So international operations is slightly harder. However, from our perspective if you’re a UK vessel, so you’re either owned by a British person or company or you’re flagged to the UK, you can come to us and we will go through an exemption process to give you a UK load line exemption.

That is almost the current badge of we have something to show we’re safe. It takes time because it’s a new process. We’re aware that that sometimes causes headaches for people but there is an option to do that. But the purpose of updating the regulations is to get around it being an exemption and actually have something that’s slightly more sturdy, that’s not the right word. Don’t use sturdy.

Sean: Something in place, yes. Something in place.

Katrina: Something in place, yes.

Sean: Because obviously there are drone aerial vehicle regulations which I know a little bit about and obviously line of sight is a really big one for a lot of that which obviously seems to be got round in certain ways with various military craft but that’s a completely different podcast. One of the things that kept coming up in the conversation that we had was about the idea of sustainability. I was bringing in my bits of sci-fi knowledge and wondering about sailing ships and things and using true sustainable and renewable energy. Do either of you know anything about that? Is there anyone working on that because that really interests me.

Katrina: I know a little but not in any technical detail. So I am aware that there are vessels that use wind to assist with their propulsion but they’re still in the early stages. I believe companies, Kongsberg are possibly looking at this but I may have that wrong. So there’s definitely companies looking at wind and how it can assist but also within the MCA, we have a team that are specifically researching the different future fuel options for exactly this reason. So what are the greener options out there not just for autonomous ships but for shipping more generally because it’s an issue that has to be addressed.

Sean: There was discussion about parking your electric ship just next to one of the wind turbines, charging it up. I mean not to put down what the guys were saying but it sounded a little bit farfetched to me, thinking about how life at sea is. But I mean is that a possibility, Jon?

Jon: There’s a lot of work going on. The university, MCA and many others and Ocean Infinity have been involved in a lot of the work recently around clean maritime and the clean maritime demonstration programme that’s been run in the UK. There’s an awful lot of work going on around how to bring that in. As part of that there is then also the question of where autonomy sits within that but it’s much wider than autonomy. So it is about the future fuels. It’s about could you plug in your ships?

Well yes because that’s already being done on portside. Could you go alongside a wind farm and plug in? Good question. Not at the moment because regulations would ensure that you couldn’t go that close to a wind farm at risk of endangering it. But many of these concepts and these ideas that yes, they sound futuristic at this point in time, may no be in five or 10 years’ time. 10 years ago autonomous ships was a very futuristic concept but here we are talking about it.

Katrina: Yes, I completely agree. I was going to say this actually, when I started in this role in autonomous shipping maybe five years ago, I would encounter people that were like this won’t happen in my lifetime, whereas you talk to hem now and they’re like all right okay, I can see how this is working. So actually those farfetched ideas will happen much more quickly than we realise. I think that’s the biggest challenge for us is the pace at which this is all happening, whether it’s the future fuels or the autonomy itself and it is happening.

Jon: I was going to say, to pick up on the discussion from Manuel and Pedro of the fact that Ocean Infinity are bringing in 85 metre remote supervised moving towards autonomous vessels, five years ago that was not a concept that people would actually think would be on the water shortly.

Sean: I did mention in the conversation about SpaceX and these autonomous drone ships that the rockets land on. I think it got taken sideways into is it a bit like being in space rather than my question which was probably more about the fact that SpaceX seemed to be doing this drone ship thing. I mean how closely are those supervised? I mean is somebody stood there with a remote control driving the ship under the rocket? I don’t know. I mean is that something either of you have got any ideas on?

Katrina: So I’m not aware of the details of them. As far as I were aware, they weren’t necessarily in the same category as an autonomous ship but they are definitely part of what’s potentially a grey area between a conventional ship, an autonomous ship and whether it becomes a platform of some form versus a conventional ship with higher tech to support those on board. So there’s definitely this area between what we see is a ship that rocks up to Southampton in the ports and what we’re talking about here and then the autonomous stuff.

Sean: Yes, I mean that’s classic regulation side of things isn’t it where define ship, define autonomous, etc.

Katrina: Don’t get me started me on that, okay. That’s a whole PhD thesis in that alone, just the define ship.

Sean: Okay, this is completely off topic, when does it stop being a boat and become a ship?

Katrina: Couldn’t tell you.

Jon: Good question. It depends on your point of view.

Sean: Well look, joking aside, we’re talking about vessels that can be larger and larger. We just mentioned 85 metres. I mean some of these, we discussed slightly about containerships and things but the craft are getting larger and larger and larger. There’s got to be safety and security issues that we have to think about. I mean we’ve already heard stories, and obviously they’re not just stories, of piracy in various seas and problems like that. What about weaponisation of these ships? What about just simple security or safety of one of them crashing into a dock? There’s a lot to think about here, isn’t there?

Katrina: So that’s from our perspective in terms of ensuring the regulations are right, is making sure that they are safe, making sure that if there’s not a remote operator, in the same way you expect a remote operator to be trained and not to crash it into the dock, we will need to have the same approvals and safety mechanisms in place for a piece of software that’s doing that so that is part of it. In terms of the other half of your question, that’s a risk but we have very strong security regulations that will also apply and that we’re amending to make sure they do apply to these unmanned autonomous vessels.

Sean: I did read somewhere that actually it might be more difficult for piracy because if there’s no crew quarters, if there’s no obvious way to make the ship but I mean I suppose cyber-attacks do happen don’t they as well. I mean all of these things need to be considered.

Jon: It will depend on the job that they’re going out and the reason they’re out there. This leads into some of the original questions around the trustworthy. If you have a vessel that has got sensors on board and is out doing a survey, you have to be able to trust that data and understand that it hasn’t been interfered with, that it is the data you originally recorded and then all the associated validation and verification of that data on the entire journey with that data back to the people that need to see it.

So from a piracy perspective, it may change from taking over the vessel and taking the vessel to wherever to climb on board and interfere with the systems. Is that potentially as big a problem? Well it could be and it depends on why they’re out there and what the reason for the vessels are.

Katrina: I think that’s an important point because part of this also is potentially how people might use these vessels as a form of getting from one country to another, not with anu particular terrorist intention but simply as a form of transport. They’re not designed for that and some of them aren’t even a traditional shape of a ship but if you’re desperate, these options are out there. So they’re all other factors, both within the UK and internationally, we have to consider within the regulations and how you deal with that.

Sean: We discussed briefly how when people start using automation, they expect it to do a better job than a human. But there’s years and years of sea experience in a lot of skippers out there. I mean how is AI supposed to cope with trying to be as good as that or better?

Katrina: From my perspective and from a regulatory view, anything we create, anything we say will be to the same standard as full conventional vessels. We shouldn’t be gold plating or expecting a high standard. However, you raised a really interesting point about the equivalent of what is machine learning but that presents problems not just for maritime as far as I’m aware and I’m not an AI expert, it’s a dilemma for all AI, this idea of machine learning and how you cope with it.

But part of that is also that learning a skipper has possibly prevents a lot of accidents without them being reported as accidents. So it’s actually how you get that learning and how you deal with those elements as well and actually does that make the AI better or worse because there’s a factor that actually it can’t take into consideration.

Sean: Yes, how do you train them to see that there’s a swell up ahead and actually let’s steer around it.

Katrina: Yes.

Jon: I think that’s one of the really key parts to the question of AI is what is the data you’re bringing in, or if you’re trying to use other techniques for the systems to learn as they’re observing, what is the data and how are they achieving that. Then that also leads back into validation and verification of if it is learning from data that’s being observed under way, how do you then validate the system that you sent out is the system that’s coming back. Was it learnt and how has it learnt it?

But yes, there’s massive questions around that about original data. I think Katrina raises that interesting point around the skippers that are out there and the crews, they have an inherent knowledge and learning and how on earth we capture that into training data is a really intriguing question.

Sean: Often these crews are working in extreme, well they’re teams, aren’t they? Often if it’s a very tightknit team, you don’t even need to say something to your colleague for them to know what they need to do next. It just happens almost automatically and yes, capturing that must be very difficult. Speaking of those crews though, it is an incredibly dangerous place to work at times, the sea, isn’t it. I mean there’s a real benefit to automating some of this stuff sure.

Katrina: Yes, hugely. For us the phrase that often comes is taking humans out of the dull, dirty and dangerous jobs and that almost feels like an obvious step to take. There’s also the actual taking people and I believe, I think Manuel and Pedro mentioned this, about that work-life balance and actually being able to have a family when you’re at sea and actually seeing your family perhaps isn’t the most conducive way of doing it.

Bearing in mind my dad went to sea and I have a very good relationship with him so it’s not us any harm. But actually some people won’t go or look at a career in maritime because of these factors. Actually these are opportunities to get different people in to the sector as well.

Sean: Perhaps open up working from home into the maritime sector.

Katrina: Maybe, yes.

Jon: The comment I’d make on automation is many of the vessels are already highly automated. Much of the engine rooms for example are automated and the chief engineer and the engineering crew are actually on board the vessel but from outside of the engine spaces, monitoring what’s happening. So there’s a lot of automation on there already. In terms of moving them then and the wider crews then off the vessels, yes, it introduces all sorts of questions around communications, etc, but there is also some of the ongoing maintenance of the vessels themselves that are going to come in and place.

That’s probably a question for why we’ll see some vessels but not all vessels moving towards the fully autonomous aspect yet because the maintenance question, there isn’t a good answer to it yet.

Sean: Just slightly tangently but hopefully you’ll realise why I’m saying this. I used to work in television station automation. What happened at first was you’d have a gallery full of people, maybe eight or 10 people all ding a very specific different job, working as a team, one might be the job of doing the lighting. One might be the sound operator, whatever. Then gradually this reduces as the automation kicks in whereas one person presses a button, then various of those things happen.

What I saw happening in that industry was that move from that to one person overseeing five, 10 channels and the automation working but the person just making sure everything was ticking along. Can we see that kind of same idea happening in shipping where maybe somebody’s watching a fleet or are we talking pie in the sky do you think? Are we way off there?

Jon: I think that’s going to be an interesting question to observe as we go forward. Certainly, and some of the research I have been doing has been looking around reducing the number of people that need to be involved directly, particularly in that oversight aspect but then there is the question of at what point is there operator overload and that’s where certainly there is a lot of ongoing research and continuing research into he human factors component which is looking at exactly that question; what can people look after? How much information can we deal with and still remain accountable for any decision that has to be made around that. If you’re looking over a fleet, it becomes a challenge.

Sean: Also the danger aspect is there. I’m not even joking that there is a phase in television that people often say if something goes wrong, don’t worry, it’s only television, nobody died. Whereas in this instance, that’s a real life possibility.

Katrina: Yes. For us, we’re watching and listening to the research that’s taking place to be able to make those decisions as to whether we will, as a UK flag, accept someone monitoring more than one vessel because for us, the biggest question is what happens when something goes wrong and then what happens when two of your vessels that you’re monitoring go wrong? How do you actually deal with that situation and what is safe to deal with.

Sean: We’ve run out of time for this particular recording. All that remains is for me to say thank you o our panel for today. So thank you, Katrina.

Katrina: Thank you. It’s been a pleasure.

Sean: Thank you, Jon.

Jon: Thank you. It’s been a pleasure as well.

Sean: If you want to get in touch with us here at the Living With AI podcast, you can visit the TAS website at www.tas.ac.uk where you can also find out more about the Trustworthy Autonomous Systems Hub. The Living With AI podcast is a production of the Trustworthy Autonomous Systems Hub, audio engineering was by Boardie Limited. Our theme music is Weekend in Tattoine by Unicorn Heads and it was presented by me, Sean Riley.

[00:56:30]