Setting Course, an ABS Podcast

Autonomous Navigation Research with Texas A&M

• American Bureau of Shipping • Season 1 • Episode 20

Share episode

As autonomous technologies rapidly evolve, institutions like Texas A&M University are pioneering research into how the technologies can be virtually verified and validated before they are deployed on real-world maritime vessels.

This special episode of Setting Course, an ABS Podcast, was recorded at the ABS Laboratory for Ocean Innovation at Texas A&M in College Station, Texas, where the university and ABS are researching the implementation of autonomous navigation — just one of many research collaborations between the two organizations.

In this episode, Texas A&M Professor of Mechanical Engineering Siva Rathinam and ABS Senior Technology Engineer Denny Raymond joined host Brad Cox to discuss the challenges, benefits and safety concerns of autonomous systems, as well as how the ongoing research can help push the industry forward.

Share this episode and leave us a review or a rating on your podcast platform of choice. You can contact the show at podcast@eagle.org. More episodes are available at www.eagle.org/podcast and all major podcast platforms.

Key Takeaways

  • The transition to autonomy will change the roles of seafarers but will be a gradual change.
  • Legal frameworks need to adapt to accommodate autonomous ships.
  • Safety concerns are paramount in the development of autonomous systems.
  • Virtual testing can accelerate the development of autonomous navigation technologies.
  • Research is needed to understand human-autonomy interaction better.
  • The future of maritime autonomy is promising and requires collective effort.

Guests

Dr. Siva Rathinam is a Professor and Industrial Advisory Committee Faculty Fellow in the Texas A&M Department of Mechanical Engineering. His research interests are in autonomous ground and aerial vehicles, sensor fusion, vision-based control, algorithms, intelligent transportation systems (ITS), robotics and artificial intelligence (AI).

Denny Raymond is a Senior Technology Engineer at ABS. With nearly two decades of experience, Denny has a multi-disciplinary background in marine product integration, power generation, dynamic positioning, and maritime autonomy. Having worked for the Saint Lucia Electricity Services, Cummins, and Basler Electric, Denny has specialized knowledge of marine engine applications, power systems and controls. Denny received his Master of Science in Engineering (Industrial/Systems Concentration) from Purdue University and his Bachelor of Science, in Mechanical Engineering from the University of the West Indies, Trinidad.

Brad Cox (00:09) 
Welcome to Setting Course, an ABS Podcast, where we're charting the future of the maritime and offshore industries. I'm your host, Brad Cox. And given where we're recording today, I really should be greeting our listeners with “howdy.” The show is once again on the road as we're recording at Texas A&M University in College Station, Texas, where everybody says things like “howdy” and “gig'em” and “hullabaloo.”

I could probably spend a couple episodes talking about Texas A&M traditions. I went to school here. I know there's a few people at ABS that went here. But really we're here to talk about the important research that's helping the maritime and offshore industries advance. ABS and Texas A&M are working together on a wide range of research initiatives dealing with all the major challenges facing our industries.

In fact, we are sitting in the ABS Lab for Ocean Innovation right here at Texas A&M. And, more specifically today, we're going to be focusing in on some of the university's research around autonomous navigation for maritime and what that can mean for the industry. 

Across from me is professor Siva Rathinam, Texas A&M Professor of Mechanical Engineering and autonomy project league.

Thank you for hosting us, Siva. 

Siva Rathinam (1:13)
Thank you. 

Brad Cox (1:15)
And next to me is ABS Senior Technology Engineer and autonomy SME, Denny Raymond. Thank you for joining us, Denny. 

Denny Raymond (1:22)
Thank you. It's a pleasure to be here. 

Brad (1:24)
Kind of setting the stage here — a lot of companies out there are working on their own autonomous navigation systems. They have their own R&D departments doing stuff like that. So what are class societies and universities doing to support that development? Denny, you want to take that one? 

Denny Raymond (1:38)
First off, maritime autonomous navigation is a disruptive technology, and it brings considerable benefits to stakeholders, for instance, the benefit of reducing the cognitive burden on seafarers and improving operational efficiency, as well as improving safety overall for the ship and vessel operations. So these are some of the reasons why stakeholders are interested in maritime autonomy, but autonomous navigation brings multiple challenges. For example, the high costs and potential risks associated with real world testing of full-scale vessels. 

And also we have different perspectives on how we look at things from a class standpoint. And how the verbiage goes in terms of taxonomy and what a fully autonomous vessel means versus non-autonomous or semi-autonomous. At ABS, we look at autonomy from a function-based perspective. So we have individual functions that we classify as autonomous. For example, navigation is just one of many functions, but a vessel may have a different function, such as an autonomous crane system or autonomous anchoring that fits in a different category of autonomous.

We issue notations for each autonomous function separately. So we don't look at the fully autonomous vessel. That is something that's being done by the IMO. The IMO has a Maritime Safety Committee and they're developing the MASS code. The MASS code looks at four degrees of autonomy from very simple degree one autonomy level, which is just a vessel that's supported by smart systems, all the way to degree four, where a vessel is fully autonomous and it could even be uncrewed.

So that is the IMO perspective and class societies are really focusing on this as a function-based technology.

Brad (3:21)
And, so, Siva, from the university perspective, what are your two organizations doing together to support this? 

Siva Rathinam (3:27)
So, marine autonomy has been in development throughout the world for the last 10 years or more than 10 years. But we have a lot of rich literature in ground autonomy and aerial autonomy. Ground transportation has been in the news for a long time. Drones have been there in the news for a long time. And we have lots of core technologies that have already been developed. So what Texas A&M is doing is to leverage all this rich literature and work with ABS to figure out what the critical issues are in marine autonomy and then design, do the entire spectrum that is to figure out, do the software/hardware-in-the-loop simulations to verify, validate each aspect of marine autonomous navigation functions. That's what that's where Texas A&M and ABS are working towards.

Brad (4:11)
So, of course, anytime you talk about autonomous ships, people are going to be asking, like, what does that mean for me? What does that mean for my job? So I think the really big questions to ask here are, why is the industry developing this sort of technology and what's going to be the expected impact on the seafarers?

Denny Raymond (4:29)
So MASS brings a lot of benefits to the industry, as I iterated earlier, and MASS technology has been proven and it continues to improve. Legal frameworks, however, require crews to be on board because these laws were developed at a time when uncrewed ships were just non-existent. Maritime laws will need to be updated to accommodate autonomous ships.

The legal community will need to get to a place where they feel comfortable with the maturity level of the technology. And the understanding that autonomous ships are just as safe as crewed or non-autonomous ships. So there's some apprehension among various stakeholders, but the technology is continuing to improve and regulations and requirements are becoming more mature.

So roles will change. Seafarers will definitely have to be more trained to be more familiar with advanced automated systems. And vendors will need to ensure that their systems are user friendly for the operators on board the vessels. The IMO has a position on crewed versus uncrewed vessels. And according to the IMO Maritime Safety Committee, in a situation where there is a human crew on board, the master needs to be on board to ensure the safety of persons on board and to exercise the master’s overriding authority. 

Also, the IMO has a position as it relates to uncrewed vessels, where every MASS has to have a remote control operation center that is assigned to control it. And there has to be a master. The master may not be on board, but the master may be at a remote operation center, whether that is a land based remote operation center or the remote operation center is on a different vessel, but there has to be a master in charge of the vessel at any given point in time.

So these are some of the ways in which the roles are changing because the master of the vessel may not necessarily be on board, but maybe operating the vessel from a remote control center. 

Brad (6:20)
So what's kind of the driving factor behind this? Is it sustainability? Is it crewing shortages? Why are the companies really looking at this?

Siva Rathinam (6:29)
Just to add to what Denny said, there are two points I wanted to make with respect to the role of the seafarers. Qualified seafarers are still going to be important even when we have autonomous systems. So their roles might be a bit updated. So for example, these seafarers could probably play more of a supervisory role in ships.

They may have to manage these autonomous systems and interact with these autonomous systems. That's one type of role I can see. Another type of role I can see is where you still need qualified seafarers to operate the ships from a remote place. So to see the ships, to understand what's happening and control these ships, so we still need qualified seafarers, but in different roles. 

Now, another reason why I think this is very important, and this is also a reason where marine autonomy kind of differs from other autonomous systems like ground and aerial, is that studies show that there is going to be a lot of shortage, crew shortage, in the future, in the near future, in fact, by 2030 and so on. So when you have crew shortage and it's going to really affect some countries even more than other countries. When you have something like that, I think autonomous systems are naturally going to be part of your solution. 

And in addition, I think autonomy, if you look at the accident data. Most of the accident, majority of the accidents are due to some human error. It's because — it's not that humans cannot do the work. It's just that they get tired. It's very challenging, difficult work. So this mundane monotonous kind of work can be easily taken over by autonomous systems and at least part of the duties can be easily automated.

So cost, safety, crew shortage, all these are reasons why autonomous systems are going to be part of it. And also there are emission regulations that are coming. So, we want to reduce emissions and things like that. So there are a lot of reasons why marine autonomy is going to be an integral part.

Brad (8:20)
Has there been research into the blending of those roles where you have the autonomy and the human in the loop. Has there been research into ways to maximize the impact of that? 

Siva Rathinam (8:34)
That's a great question. I feel like marine autonomy is more in a nascent stage. We are not there yet. So, I've seen use cases and studies like this in different places like in Europe and other places, but we need a lot more work. We need a significant amount of work is needed to actually understand this interaction between humans and autonomy because, I was going to talk about later that, you know, you have different types of vessels, cargo ships, you have different types of waters, weather conditions, environmental conditions. And you are dealing with international waters with different jurisdictions. So I think the way it's going to progress is use case by use case, and we do have some autonomous systems like companies have developed some autonomous systems, but still a lot more work needs to be done. So classification societies like ABS has to come in and everyone has to come together to make that happen. 

Brad (9:27)
Siva, I know you mentioned the decarbonization aspect. Denny, how can autonomy impact that? How can that help support sustainability goals? 

Denny Raymond (9:37)
Maritime autonomy can help improve efficiency in several different ways. In the case of autonomous navigation, these autonomous navigation systems can enable vessels to chart courses that are more optimized, and as a result, reduce fuel consumption. If fuel consumption is reduced, then carbon emissions are going to be reduced. And as a result, that can help towards the achievement of net-zero targets.

In addition, besides just autonomous navigation, optimizing other processes, using autonomous technologies and using machine learning and AI, for instance, optimizing the machine processes and just how different operations of the vessel function, it is possible to improve efficiencies and just help to reduce emissions overall.

Siva Rathinam (10:22)
Just to add to what Denny said, autonomy can lead to repetitive, reliable and consistent operations emissions. Studies have shown that if you have long routes for the vehicles, then optimizing the paths of the ships or controlling their speed profiles can actually provide up to 10 percent of fuel savings, or in fact more than that. And that can definitely lead to reduced emissions. 

Brad (10:49)
Of course, we talk about fuels and stuff on the show all the time, and safety always comes back around in that conversation, of course. And, any technology, you're going to be concerned about safety. It's certainly a top priority for us at ABS. When you look at implementing these autonomous systems, what are the safety concerns and has research revealed any unexpected risks? 

Siva Rathinam (11:09)
Right now, almost all the ships or most of the ships are controlled by human, human operators or seafarers. It's not like we're going to suddenly get into autonomy. This is not a switch on, off switch. This is going to be a long process and there is going to be a gradual transition from human control to autonomous control. So, which means that we can expect over the next decade or so, or more than that, the transitioning between human control to autonomous control is very important.

And that's the reason why Texas A&M and ABS worked on a project. The first, the first phase of the project was exactly on this. We want to understand this transition better. So we want to look at what critical factors actually affect this transition. 

We have done a very detailed study, a literature review. We have, in fact, we listed all the critical factors that depend on system characteristics, environmental characteristics, and seafarer characteristics that affect this transition. And I'm not sure we were surprised, but we were definitely — understood that this is a very complicated problem. This is a very challenging problem. 

What we did was to look at the IMO accident database. So we looked at more than 1,200 reports and then we narrowed that into 18 reports that were more closer to failures with respect to autopilots because we don't actually have autonomous systems on board. So it's the closest we can get to. And we based on that — there is a framework that we worked on called the system theoretic approach where we take all these accident reports. We look at each of the transitions that can happen between human and autonomous systems. We look at the critical factors and what scenarios could possibly cause it.

And then we rank the critical factors based on the impact or the loss that can lead to the environment, to the companies, to the government and all that. And when we ranked it, we found that the seafarers navigation experience, their cognitive performance, their familiarity with automation and also the reliability of the system are the critical factors. At least in, you know, in this limited study that we did. So it's not really a surprise, but we just found a lot of challenges. And we also realized that in order to do this work, we have to create different types of environments. We have to simulate different weather conditions, different ships and so on. And that was basically the motivation for the phase two of the project, which is what we're working on currently. 

Denny Raymond (13:30)
And just to add to what Siva said, so we foresee a time in the near future where humans will have to work together with these advanced software systems for autonomous ships. It's not going to be a sudden jump towards fully autonomous vessels. In terms of the critical factors that were identified within phase one of the project, we think that these can be used to help with risk assessment and to develop requirements that are stronger and requirements that are mature and relevant for the industry. So we're excited about phase two of the project, which is the development of a virtual testing framework for verification and validation of autonomous navigation.

And we think that with that framework, we can generate some of the data that we were not able to generate in phase one of the project, and we can use that data and also use some of the learnings from phase one of the project in terms of the critical factors and the cause scenarios when transitioning from autonomous to manual control or vice versa.

You can think of a scenario where a vessel is navigating through the open waters and then it comes into a port and now it is time to transition from autonomous control to manual control. What are the critical factors that can lead to an unsafe incident, not only human factors, but also factors related to sensors, machinery, software, equipment. We've sort of looked at this holistically and that's how we've determined what these critical factors are. 

Brad (14:56)
So, obviously that gets into a lot of this specific research that ABS and Texas A&M are working together on. So, Siva, can you go into a little more detail about, you know, what exactly is virtual testing and, you know, how this can apply here and what the impact is going to be?

Siva Rathinam (15:10)
Virtual testing is really important, especially if you are developing a new system. The real world scenarios are complicated. We can't test — every time we have a new algorithm, we cannot go to the real world and test everything and, so, we want to make sure that the algorithms are verified and validated in simulations before we go to the real world.

That's the reason why a verification simulation framework is quite important. Now, in a simulation framework, one of the key components of it are the sensors. Because the sensors on a ship are the eyes of the ship. They see the environment, they sense the environment, and you get data from the environment, fuse it and understand where the ship is, the situational awareness of the ship is.

So that, the sensor and the sensor fusion algorithms are an integral part of simulation framework. So once you get the sensing data, then you can understand how — what is going around the ship, and then you can plan the paths for the ship and also control the ship, like maneuver the ship, then turn left, right, go straight and what speeds they should go on.

In addition to the sensing, sensor fusion and planning and control modules, you also need to model the ship, basically model the dynamics of the ship. And also we need to model the environment in which the ship is going to operate in. So, it's basically what type of waters. Is it near the coast? Is it port, narrow passage? Is it ocean? And also the weather could also be a huge thing, whether cover is there, whether cloud cover is there or not there. We simulate all of these things and try to at least create a very realistic environment for you to test the algorithm. So when you develop, let's say a control algorithm, a sensing algorithm, all that — if you have the whole framework, you can put it there and you can run the whole simulation closed loop and verify if the performance that you get matches with what you want.

If not, you can change things, you can change your algorithms, you can change your models to verify and validate each of the functions of your autonomous system. So this is something very standard, very basic we do for any autonomous navigation. 

Denny Raymond (17:13)
And standards and requirements for certification are still in a developmental stage because maritime autonomy is still so new and real-world testing is very costly. It's logistically complex, it can present safety challenges, trying to test autonomous functions on full-scale, real-world vessels. That's very difficult to do, and it can take a long time. 

So virtual testing is really an alternative to real world testing that can help accelerate the development of knowledge that is required to develop rules, to develop requirements, to develop standards so that we can certify these autonomous systems, we can verify that the algorithms work correctly and that they will allow for safe navigation. 

Part of what we're doing with the project is to look at various collision scenarios and, Siva touched on it regarding extreme weather conditions, extreme environmental conditions. And we basically want to pressure test these algorithms that vendors are providing so that we can certify them and give a level of assurance that these systems are going to perform safely and give the confidence level to the industry that this is being conducted in a safe manner.

And the whole idea is to give a sense of confidence that autonomous navigation is going to be just as safe and even safer than navigation of non-autonomous vessels. And that is where the goal-based standard comes in because it provides a mechanism to show that you can achieve equivalence where you have an autonomous function.

We don't have prescriptive requirements for those. And because it's all software based, it's very fluid. It's very difficult to develop prescriptive requirements for these kinds of systems. And so we can demonstrate through verification and validation of software. That these advanced systems can meet the intent of a vessel that is non-autonomous, more of a traditional crewed vessel.

Brad (19:04)
Great. So, kind of taking a step back, obviously, the research A&M is doing here has a role in supporting the industry and supporting innovation. So, Siva, how important is the role of academic research in pushing the industry forward? 

Siva Rathinam (19:18)
I think it's very important because we have, as I told before, we have rich literature in ground autonomy and aerial autonomy. Even though marine autonomy might look different and it is different because applications are different, the rules are different, conventions are different, but the core modules that make up the autonomous navigation, sensing, control, planning, dynamic modeling, and all that, that I talked about a few minutes back, they're all pretty much the same.

So if we can learn from what we have done before for other modes of transportation, what we want to do, at least Texas A&M and ABS wants to do is to learn from it, tailor it and adapt it to marine automation. So I think that's where we come in. So we have a rich history, the literature, we can look at it and we know what things have worked and what things have not worked and we can make the marine automation more efficient.

Denny Raymond (20:07)
Yeah, universities are well positioned to help class societies like ABS to advance these new technologies, because there's limited data when you have new technology, limited real world data that we can use to actually quantify and understand the risks involved. So with the research that Texas A&M is doing for autonomy and other projects that they're helping us with, we're able to better understand the level of risks, quantify the risks, and so develop requirements that actually address those risks in a practical way. And we can be effective in doing that. 

Brad (20:40)
So, as we're kind of wrapping up here, just wanted to really kind of open things up, let you guys, if you have any closing thoughts for our listeners, you know, anything you want to leave them with. So, Siva, you want to go first? 

Siva Rathinam (20:50)
To me, it's an exciting time to be part of the development of marine autonomy. You know, I've looked at a lot of these, a lot of studies and reports from all over the world. And it's predicted that just the ship autonomy market is going to be more than 150 billion dollars by 2030 for many reasons.

And so it's going to be there. Autonomy is going to be there. So if you look at most of the current, as I told, most of the current ship navigation, everything is controlled by seafarers, right? And so all the conventions and the rules were written with human in the loop. But now things are going to change with autonomy. Things are going to change, which means that all these rules and conventions have to be rewritten or at least looked at and adapted to where we're going to go. 

And so this also implies that everyone has to come together, the classification societies, the universities, researchers, governments, private industries, all of them have to work together and consider use case by use case and figure out how we can move this forward. So I think it's pretty exciting and I think a lot of innovations can come from it. 

Denny Raymond (21:53)
The maritime industry is on the verge of a new wave of technologies and autonomy is just one of those. So it's a great time to be part of this, to be doing this type of work. As Siva said, universities, class societies, all the stakeholders need to come together, shipyards and vendors, manufacturers all need to work together so that we can implement these technologies safely. It's a great time to be part of this industry and we're excited to see what the future brings. 

Brad (22:19)
All right. Well, this was fantastic guys. Really interested to see how autonomous systems develop and how we can work better with these systems. Once again, thank you, Siva, Texas A&M for hosting us here. And thank you, Denny, for joining us on this episode. 

Siva Rathinam (22:33)
Thank you. Thank you for, I mean, it's a great opportunity for Texas A&M and ABS to work together in this. We are all excited in this and looking forward for future collaborations. 

Denny Raymond (22:42)
Well, thank you. And thanks Siva for working with us and recording this podcast with me. It's been a pleasure. 

Brad (22:47)
And for everybody listening, thank you for joining us for another episode of Setting Course. Be sure to subscribe, leave a review and share this episode. To learn more about the innovative technologies impacting the maritime and offshore industries, visit us at www.eagle.org. Thank you for listening.