Unpacking the AAQG Auditor Workshop

In this episode, we are joined by Brett Cooper, the Associate Director of Quality System Compliance at Raytheon Corporation and the newly appointed Certification Oversight Lead for the Americas Aerospace Quality Group (AAQG). 

Join us as we dive into the important work of the Certification Oversight Team in the Americas, which manages the IAQG certification scheme.  

Exploring key topics such as: 

• The significance of the 9145 standard and its impact on auditors. 
• The game-changing OCAP (Organization Certification Analysis Process) and its role in audit planning. 
• The upcoming IA-9104-1 performance-based surveillance and recertification process, which rewards organizations for meeting performance standards. 
• The importance of understanding requirements and effective change management in audits. 

Click on the link to listen to the supplemental webinar on the 9145 presentations https://www.youtube.com/watch?v=DB8JGcG_O7k.  

 To learn more, visit https://iaqg.org.   

Transcript

Quality Horizon Podcast – Unpacking the AAQG Auditor Workshop 

Susan Matson: [00:14-01:03] Greetings everyone and welcome. I'm your host Susan Matson, and with me today is Bret Cooper. Bret is the Assocaite Director, Quality System Compliance for Raytheon Technologies, and recently became the Certification Oversight Lead for the AAQG. Welcome to the show Bret. So I wanted to, before we get started, Bret, I wanted to remind our listeners that the IAQG is broken into three geographic areas. The APAQG represents Asian Pacific countries. The EAQG represents Europe, the Middle East, and Africa. Ans the AAQG represents the Americas. And them within each of them, there is a sector specific certification oversight team. So, Bret, before we get into what we wanted to talk about today, can you just give our listeners a brief understanding of what this team does?  

Bret Cooper: [01:03-01:35] Sure. So in the Americas, we have a certification oversight team. It's a committee made up of aviation space and defense OEMs, representatives from the Americas sector. The team's primary charter is to manage the Americas IAQG I-COP certification scheme within our sector. by conducting oversight assessments of the AVs, CVs, and AAVs and documenting the results of those assessments in the OASIS database system. 

Susan Matson: [01:36-01:51] Okay, so it sounds like auditors are probably part of a central focus in what you do, and helping them stay current in the Americas, your team hosts an annual auditor workshop. So can you tell us what happens during these events? 

Bret Cooper: [01:51-02:23] Yeah, so during the events, we pull together a list of speakers to provide information and education to the auditors within our sector. It's typically a day to a day and a half event. We open it up to all auditors, and in the end, we give them continuous professional development learning hours for their participation. We've found this to be very successful in our sector. There's an appetite for it, and so we continue to host these events annually. 

Susan Matson: [02:23-03:03] Thank you. Now, I'm understanding that these are geared to the Americas, but I'm sure that most all the topics that are covered do have some interest points and insights that would hold interest for anybody, regardless of where you are in the world of auditors. Let's kind of go through what you did in this most recent one and share some highlights and news for the listeners. So first off I wanted to do was talk about your keynote speaker. So Lori Gillespie with the ANAB, she was your speaker at this past workshop and she discussed improving stakeholder confidence. So can you share a little bit about what she covered and why it was important from your perspective? 

Bret Cooper: [03:04-04:06] Yeah, I think Lori took the opportunity to kind of unpackage her presentation and really focus in on the auditors that are auditing within our sector. And it really is a more broad activity as we think about it reaching across the globe. But auditors really need to recognize the importance and the role that they hold in the certification process. Customers who are issuing contracts across the world trust what it means to be a certified organization in our industry. And Lori took the time to really remind the auditors in attendance of the importance of the work they do, the value of the certification and how it's critically important to everyone involved and why we continue to do the work we do with auditors and training. And so, yeah, it was a really good way to kick off the workshop and kind of set the tone for the week on the importance of stakeholder confidence and what it means to be a certified organization and the role that the auditors play. in that process. 

Susan Matson: [04:06-04:41] I bet that really was. And I'm sure, as you said, totally set the stage for the rest of it. But diving right into some of the meat of it, you covered the 9145. And while it's it's been around, auditors are now to include that when it's contractually flowed down. So there was a whole discussion and highlights about the shall statements in the 9145. And I know that there was some discussion and they went into detail, the presenters, about that. Can you help people get highlights of what that meant and where they might be able to learn more about it? 

Bret Cooper: [04:42-07:18] Yeah, so the team, they did an outstanding job, by the way, of really kind of pulling together what some of the, I would say, the key shall statements were. There's several shall statements in 9145, but they highlighted 10 of them. I'm not going to talk about all 10 of them, but we can certainly take a moment here to talk about some of the key ones that I think were a takeaway for everyone. The first one being around APQP and planning for project management, right? As an example, they took the time to really express upon the auditors the importance of identifying who the project owners are and what their responsibilities will be as it relates to completing project objectives, ensuring that dates are planned, schedules are set to align with customers' needs. It's important that that becomes kind of the foundation and kickoff for the 9145 activity. They took some time to talk about design-related deliverables and what to look for as part of that activity. Some of those examples included design documentation, bill of materials, design risk analysis, special requirements, and key characteristics, just to name a few of the things. The team also discussed and encouraged auditors to make sure they're reviewing key artifacts like the process flow diagrams, the PFMEAs, control plans, work instructions, The MSA plans around measurement, production readiness reviews, first articles. There's a lot to unpackage when we think about 9145. And so this team did a great job, you know, reviewing these. you know, these activities and things to look for and listen for as part of that training session. And as far as additional resources, I know you had asked about that. You can go out to the IAQG website and look at the Supply Chain Management Handbook. There you're going to find general information and references on APQP. downloads that you can use to kind of help provide guidance whether you're a certified organization looking at it from a planning perspective or if you're an auditor looking at what are some of the artifacts that should be included and how would I go about looking for those things. So a lot of good information out there on the supply chain management handbook. portion of the IAQG homepage. I encourage folks to go out there if they want more information. 

Susan Matson: [07:18-07:45] Yeah, and it's my understanding that that particular presentation, they actually ended up doing a supplemental webinar, which is also on the SCMH site. So we'll put that in the show notes if anybody wants to have a full understanding of what that particular presentation was. But moving on, it's my understanding there was also a lot of talk about OCAP and how it's a game changer for the certification scheme. Can you expand on that? 

Bret Cooper: [07:45-08:59] Yeah, absolutely. I think the industry is taking a really good step forward here with the implementation of OCAP. And when we think about audit planning activity today, it's really around looking at an organization's headcount and and maybe gleaning a little bit of information in advance of an audit. But with OCAP, it's going to take in a set of data and criteria along with headcount to really determine where we should be applying our audit duration and time. It's going to be looking at things like on-time delivery, customer satisfaction, conformity of delivered products. It's also going to look at the organization's past pair performance. And depending on the overall rating that comes out of the OCAP planning tool, some organizations may qualify for a deduction in audit duration, while others may experience an increase in audit duration based on overall risk. It's really taking a big step forward from the standpoint of we are now taking into consideration performance, whereas in the past we were simply just looking at headcount. 

Susan Matson: [08:59-09:04] So how should we be looking at this as an opportunity to improve that audit activity? 

Bret Cooper: [09:05-09:50 So I think with the updates in 9104-1 and the implementation of OCAP, like I said, there's now going to be time allocated in advance of the audit where this planning activity is going to occur. We're going to be looking at the inputs of OCAP And it's really going to drive us to create more targeted schedules that focus on areas of risk within an organization. This will ensure the auditors are auditing in areas where risk has been identified and that we're really touching on the concerns that have been highlighted through performance, both internal and through our customer feedback, customer satisfaction. So I think it's a really good thing for the organization moving forward, for our industry. 

Susan Matson: [09:50-10:12] Oh, it sounds like it. I'll be curious to see how people respond to it. So let's shift gears a little bit and talk about the IA-9104, the upcoming, I should say, IA-9104-1, the performance-based surveillance and recertification process. What's this going to include, and why are some people calling it a reward? 

Bret Cooper: [10:13-12:18] Well, I would so, you know, for starters, it's it's going to be based on organizational performance. As an example, if an organization meets the requirements for qualification outlined in the standard, they have the processes in place, they have the programs to support it, they have the trained resources in place, then an organization can apply. Provided all the requirements have been met, the organization could see a reduction in certification audit activity. So, as an example, your internal audit team or your second-party audit team, I should say, would be supplemental to the certification body. So, if you were to think about, let's say, a multi-site that has six sites on the certificate, for an organization that has been approved for PBSRP, their audit activity may look something like Year one would be two sites plus the central function. Year two would be two more sites plus the central function. And then year three would be the remaining two sites plus the central function. And so, in a traditional setting, organizations today go through and have a Surveillance 1, Surveillance 2, and recertification with PBSRP. It's really replacing the Heritage ASRP program that was out there, and it's going to allow organizations to, if performance is met, if your OCAP risk ratings are good, an organization could see a reduction in overall audit duration. It's important to note, though, that as this review happens, organizations need to make sure they're maintaining their performance. So, if an organization's performance aggregates, it could lead to an increase in surveillance time. It could also lead to an organization or a site being removed from PBSRP. 

Susan Matson: [12:19-12:52 So there actually could be a reward if everything is going smoothly and going well for them. Let's talk about change management and the requirement consumptions. I know, I think we talked about this once before, right after the workshop happened, and you had identified this section as something that really had the audience very interested in it. So, you know, can we, Let's talk about maybe understanding the requirements themselves first. 

Bret Cooper: [12:52-14:44] Yeah, absolutely. And if we think about requirements, on the surface, it may sound simple. However, when you take a deeper look, there is a lot to consider. As trained auditors, the one thing we have is the standard is known, right? It's the known piece. So from there, it's really about assessing contractual requirements, including contract mods, and changes to those contracts, statutory and regulatory requirements, and of course, any other source of requirements that may be applicable based on the audit that is taking place. From there, you know, the auditors need to take into consideration what does the certified organization have as far as their own set of requirements, right, that need to be considered as well in auditing. And so I think that to kind of bring all this together and what was really a highlight from the workshop session was that there was some examples that were given during the session that I think really helped to draw conclusions for people on the importance of looking at requirements and requirements consumption. You know, as auditors, in the end, it's important to recognize that audits are a snapshot in time. And that not 100% of all requirements are typically going to be achievable during an audit session, right? So, as auditors, you know, it was really impressed upon them. They need to be able to create audit trails that touch on such things as customer requirements. covering the applicable changes, ensuring that they're in compliance with those statutory and regulatory requirements, while also, you know, making sure that they meet the planned objectives and activities that were outlined by the organization and communicated to the customer. So, it was a really good session. 

Susan Matson: [14:44-15:16] Oh, I bet. And I'm sure that those samples that were given really helped to put things into perspective. So much of what is available doesn't always apply when you're out in the world. So those examples probably helped them tremendously. So there was a lot of time spent on NCR grading and our CCA casual analysis. So a couple thoughts and ideas there that I'd love to talk about a little more. The kind of what, where are some of these controls that people can put into place? 

Bret Cooper: [15:17-16:07] So some of the controls that auditors can look for are typically falling to one of two categories. One is administrative controls or engineering controls when it comes to problem solving. Administrative controls can oftentimes be a quicker and easier fix versus engineering controls. However, with administrative controls, an organization is more likely to rely on human behavior Whereas with engineering controls, it's likely going to require more planning, take longer to implement, may even cost the organization additional dollars to fix. However, it is clear that these types of fixes are typically seen as having a higher likelihood of preventing reoccurrence. 

 Susan Matson: [16:08-16:11] So are there common issues that we're seeing with RCCA? 

 Bret Cooper: [16:11-17:21] Yeah, I would say one of the common issues that is typically seen is that they're only addressing the symptom and not the true root cause. Organizations, when corrective actions are evaluated, oftentimes they're not seen as using cross-functional teams to analyze the root cause analysis. Sometimes we see where the corrective action focuses on operator error versus the true process failure. It's also been noted and was communicated with the auditors that organizations at times don't verify that the fix in place is really the true fix. And one thing that was shared with each of the auditors is, well, does the organization go back and verify? Can they recreate the failure if they were to reintroduce the root cause that was identified? And so being able to turn the problem on and off like a light switch is something that was shared with the auditors and something to just look for as they're doing their review of corrective actions within audit activity.  

Susan Matson: [17:21-17:41] I bet that was very helpful to share. And as the whole two days of the workshop. Now, we talked a little bit about it in the very beginning of this discussion of what was included, but who puts this workshop on? And do you have them every year? Is it every quarter? How does that work? 

Bret Cooper: [17:42-19:10] Yeah, so the workshops are hosted annually. We even managed to do a virtual hybrid one during COVID. But yes, we do, in the Americas, we do our best to host an annual workshop every year. We have one planned for next July, and the team has already coming together to think about topics for next year. One of the things, you know, we'll probably do again this year, like we have in the past, is we will solicit feedback from the auditor community. I know at the most recent AAQG meeting in Indianapolis, I had people walk up, and so I've already started a list. I have three topics for discussion already that folks are interested in hearing more about for the 2025 workshop. Yeah, we're certainly looking to continue to do this. We see the benefit and the value. We get a lot of participation from the certification bodies as well. They send members from their office to glean information and take back and share with those auditors who are not able to participate. So I think the coverage has been good, the appetite has been healthy, and the participation from the presenters has always been extremely enthusiastic. So it's a really exciting couple of days when we get together. 

Susan Matson: [19:10-19:51] Oh, that's good. I'm glad to hear that. So you did say July of 2025. I did hear that. I also heard that this is for the Americas. The European, the EAQG and the APAQG do also do similar, not similar. I mean, they do their own version of keeping everyone in the know of what's going on and what are some highlights. But for if you're in the Americas, how do we, I know it's on the events, we do also the calendar events on the IAQG, I should say, and we'll be posting a save the date once we get closer to that, right, Bret? 

 Bret Cooper: [19:52-20:36] Yeah, actually, the team is working on the language that we want to use for the save the date now. And so Jed hopefully have something to the communications team to get posted for us in the next couple of weeks. We also, during our ACOT closed session meetings, we have CB focals who are in attendance, and we communicate to them as well. So, when they host their CB town halls, it's another way that communication goes out to the CBs. Subsequently, they're communicating to the auditors that are working for them about the upcoming workshop activities. So, we have a couple of ways that the communication is getting out. 

 Susan Matson: [20:26-20:56] Wonderful, wonderful. So I want you to pull out your crystal ball, and hopefully you can give us a sneak peek on maybe what are some things on the horizon from the America Certification Oversight Team? What are some things that we should be thinking that are coming down the pike? 

 Bret Cooper: [20:56-21:55] I would certainly say that everybody is sitting here waiting for the release of the trilogy of the IA-9104 standards along with IA 9101. That's going to drive a lot of new requirements. When I think about internal to our ACOT committee itself, the OP assessors who are conducting all the oversight activity, they're going to have to go through and be retrained. We're going to be now off assessing a new set of requirements that have been outlined from an auditor perspective. for those auditors who are auditing in our industry. They're going to be required to take updated 9101 training. They're going to learn about the OCAP tool and how to use that. So there's a lot of change that's going to be coming in the near future. And so I think just like the other sectors, we're sitting here patiently waiting and we'll be ready to go when the time comes. 

 Susan Matson: [21:57-22:46] Oh, it's coming. I bet. Yes. And I'm, I'm sure everybody, you know, there'll be information being posted on the, on the site, on the website. I'm sure that there'll be information that comes out and is also posted on LinkedIn. And maybe Bret, you can come back and we can talk about it once we have more information about that, or we can pull in some of the other leadership from the 9100, the 9104 series the Trilogy and the 9101, so good things still to come. Okay, so anyone who's come so far, they're listening. I know one of the questions out there that somebody's asking is, how do they get involved in this? How can they become a part of the Certification Oversight Team in the Americas, or the IAQG for that matter? 

 Bret Cooper: [22:47-23:38] Absolutely. So I mean, I would always encourage everyone to start with your member company rep first. So if you are working for an OEM in the industry, I would I would reach out to your member company rep first. If you're currently not a member company, and you're not engaged, but you've listened to this podcast today, and you're interested in learning more about the IAQG, you can go out to the IAQG website. There's a contact us feature on the website. We encourage you to fill out the request. There's a drop down for the type of request that you're actually or the information that you're seeking. Fill that out and someone will be in touch with you and kind of help provide some guidance on how you can get more involved within the IAQG and AAQG in America sector. 

 Susan Matson: [23:39-23:59] Awesome. Well, great. Thank you so much, Bret. I really appreciate the time you were able to give us so we could connect, I should say and share news about the Certification Oversight Team and what's happening in the Americas. So best of luck in your new position. And I look forward to hopefully talking again soon. 

 Bret Cooper: [23:59-24:04] Excellent. Thank you. And I appreciate the opportunity and look forward to the changes that are coming. 

 Susan Matson: [24:05-24:18] Yep, absolutely. Well, this is Susan Matson, and you have been listening to the IAQG Quality Horizon. Until next time, stay safe.