Changes and improvements to 9104-1 and OCAP!

We decided to have a sit down with Tim Lee and Darrell Taylor to talk about the changes and improvements to the 9104-1 and the introduction of the Organization Certification Analysis Process, otherwise known as OCAP.

The conversation highlights some key points and answers questions about why the improvements were necessary and what’s really changed. 

Take a listen if you want to hear about 9104-1 and OCAP’s process, concept, and how it works. 

More about our guests: 

Tim Lee 

• Boeing Associate Technical Fellow & Supplier Quality Specialist, The Boeing Company
• Supporting 9100 Quality Management System approvals, certification & oversight 
• Chair of the Americas Certification Oversight Team
• Voting member of the IAQG Certification Oversight Team
• IAQG International Accreditation Forum Representative
• International Document Representative 9104-1 “AS&D QMS Certification Requirements”

Darrell Taylor 

• Former Corporate Director of Quality Management Systems with Raytheon Technologies. Retired.
• Quality field professional with over 40 years of experience
• Served the AAQG and IAQG for 15 years in various activities 
• Former AAQG RMC Chair
• Member of the 9104-1 writing team 
• Former Americas document representative  

The IAQG is the International Aerospace Quality Group and sets the standard for quality within the worldwide supply chain within the aviation, space, and defense industry. IAQG currently maintains 26 active standards that establish common/shared tools and methods for quality improvement. To learn more, visit https://iaqg.org.  

Transcript

The Quality Horizon – Changes and improvements to 9104-1 and OCAP 

 Susan: Greetings everyone and welcome. I'm your host, Susan Matson, and with me today are Tim Lee and Daryl Taylor. Tim is with the Boeing company where he is the Associate technical fellow and supply quality specialist. Additionally, Tim is very active in the IAQG and AAQG. His participation includes supporting the 9100 Quality Management System, approval certification and oversight. Being the chair of the America certification oversight team, acting as voting member of the IAQG certification oversight team, being the IAQG international accreditation forum representative, as well as the International document representative for the 90 1041 AS and DQMS certification requirements.  

Susan: Darrell comes to us today with over 40 years experience in the quality field. A recent retiree from Raytheon technologies. He was the corporate director of quality management systems, and has also contributed to various activities within the AAQ and IAQG over the years, including being the former AAQG RMC chair member of the 9104 - 1 writing team, a former America's document representative. Welcome to the show, both of you. 

 Tim: Thank you. 

Darrell: Thank you. 

Susan: Thank you. Well, I have to say that was a long list of work that you've done to progress the IAQG mission. And on behalf of everyone at IAQG, thank you for sharing your expertise. So today's topic, however, is all about change, and specifically changes to the 90 1041 and the introduction of the OCAP tool. So Tim, my first question is to you, for those who are listening and are slightly unfamiliar with the 9104 series, can you tell us what it is and generally speaking, what's the purpose of the 90 1041

Tim: Well, thank you very much. And let me start by saying it's my honor and privileged to be here today and greatly appreciate Darrell’s contributions. And as Darrell will attest, this is a long journey. It's a long journey of change, and a number of years went into updating this very important standard in our trilogy of standards to ensure that we do have a robust, valued and compliant system of certification across the aviation, space and defense industry. So very proud of that, and a lot of people behind Darrell and I, they're working very hard to make sure that happens.

 Tim:  But back to your question, because it is important to set the stage, if you will, and clearly understand how we manage our certification scheme and manage the oversight of that to ensure that it is compliant. We have a trilogy of standards. We have three standards, with the 9104 -1 being the top of the trilogy, if you will. That is the baseline of requirements. It's requirements for our approval of accreditation bodies, how we manage oversight and manage the committees in the three IAQG sectors, not only in the Americas, where Darrell I'm from, but also in our European theater and the Asia Pacific sector. So very top level document is the dash one that contains the requirements for aviation, space and defense quality management systems. Certification of our three standards, the 9100 standard for design and manufacturers, the 9120 standard for distributors, as well as 9110 is focused on our maintenance, repair and overhaul criteria and suppliers. 

 Tim: So having that top levels dash one standard sets the stage, but we also support it by two additional standards that are very important. So in order to have a robust scheme, you need to have skilled, competent, qualified auditors and training providers, and that criteria is defined within our 9104 -3 standard. Then, of course, it's holding ourselves, as well as all the stakeholders in our very critical community, to to conformance, and that's done through an oversight process that's defined within 9104 -2. So setting the stage is very important to understand that trilogy. 

Tim: One last piece that's key to understanding our requirements. We used to use the term ISO as supplemented by aviation, space and defense, unique requirements that still holds true in certification, because we utilize the ISO criteria. ISO 17021 -1, for example, as well as the international accreditation form mandatory documents to establish our baseline. That's the baseline criteria. And we use this term, and Darryl is very familiar with it. What would ISO do? Right? Because that is our baseline and where we start, and then we supplement that by unique requirements that are key to the industry that we work in and and we supplement that through the dash one. So again, that's the overall top level executive review, if you will. So I'm looking forward to your questions. Susan, back to you. 

Susan: Thank you. So what would ISO do? And it sounds like that might be the basis or the rationale for change for the 22 update. So can you give us a little background on on how the how, how those changes that happen from ISO in the 2015 and the 2017 criteria that contributed to the 9104-1 22 , changes? 

 Tim: Yeah. Let me give a brief start. And I want to turn it over to Darrell. Get a another perspective, if you will, but, but it's, you know, we're all quality minded professionals, right? And it's all about continual improvement, looking at existing criteria, looking at ways not to just write new requirements, because it's a great idea. How do you improve certification? Well, through the international accreditation forum, they've looked over the years, and they've looked for ways to improve, ways to improve the calculation of audit duration, for example, with updates to the if mandatory five document understanding certification structure if document MD1 talks about multi site certifications. 

Tim: And just a key change is acknowledging virtual sites, for example, in our new post pandemic world, right? And then the ISO 17020-1 standard itself changed and created some additional, very strong criteria. So we didn't want to duplicate what was already in place, but we needed to align with those changes, and that's key to this major change to our standard. Darrell , anything you want to add to that.

 Darrell: Tim, you're right on the whole point was really to leverage what's already in ISO. We were telling people, when you go to evaluate this standard, you really need to sit down with 17021, 17011 as well. Because those things we did not repeat. We were pretty adamant about not repeating. If you go back to the original version, the 2012 version of as 9104 there were a lot of redundancies in there to what was already established in ISO. We took all of those out. We made sure that we didn't want we took ISO as our foundation. We're going to grow from that foundation. We're not going to change it. We're going to add to that foundation. 

Darrell: And clearly, everything that we've done here builds on what was already built into 17021, 17011, even MD five, which is the audit duration calculation we built on that we went over and above, recognized we have a lot more shell statements to deal with, so it's got to be bigger than MD five, bigger than ISO, and we incorporated all that. So that was the foundation of how we created this. A lot of work went on in the front end to understand exactly all those changes before we even started writing 9104 so it certainly is the foundation. 

Susan: Yeah, it's go ahead, Tim. I'm sorry. 

 Tim: No, that's fine. I would just want to also add, and it's really one of the things that we really identified when evaluating those ISO criteria was our new OCAP tool, right organization, certification, analysis process, we believe, putting forth the effort upfront, with good planning, with utilizing the quality management system performance as input into that process, we'll get a better output. So just an example why we're here today, and that extra supplemental requirements, that OCAP tool is very key to success in our scheme. 

Susan: So you just took some words right out of my mouth, because I wanted to talk about some of these key changes, and OCAP is is a big one. So before we go into that, because Dara, I really want to hear some of the details of how that came to be, and what are some of the things that you can now the features you can now do with OCAP. But what were some of the other changes? Just a top of mind of what you what you're seeing in the new 9104-1.

 Darrell: I would say one of the key things that we picked up on very quickly was the fact that campus and several site that actually came up as a topic during our SWOT analysis. So if you recall what we did here, when, in 2015 when we should have been thinking about rewriting this, or 2017 we were tasked with going off and doing a SWOT analysis all over the world. I don't know how many analysis we did, but we had multiple meetings in Europe and Japan, in all over America, at multiple AAQ meetings and RMC meetings. 

 Darrell: We set aside time. And one of the things that came out of it was a total lack of confusion over certain campus certification and several site certification. So one of the key changes is we decided that we're not going to be different than the rest of the world. Everybody else deals with single site and multi site. So we rolled this back to meet the ISO requirements. And that was a key requirement. That was a key change coming out of the previous version one, key change that came out of the previous version of 9104-1.

 Susan: So just to clarify that the concept of campus is not retained in the new in this version Correct?

 Darrell: Correct, you would find, if you do the calculations, you'll find a campus calculation actually violated the rules of MD five, and we were just not going to be putting ourselves. A position as an industry. We've talked a lot already about how important meeting that ISO foundation is, and we are building on it. Does it make sense that in an aerospace community with a whole bunch of additional shell statements that you would audit less time than you would if you were doing an ISO type audit? That just doesn't make sense, and you're right. Campus, several site, these were dropped, and we we got back to the foundation of single site and multi site. 

Tim: Yeah, it's all about leaning the standard and only sticking to those requirements which enhance the value of certification. Just a couple other areas that we addressed in the new standard, uh, one being, is auditor rotation, right? So it's important to us that we get a fresh set of eyes on a certified organization that, just because of familiarity and getting auditing over and over again the same processes. So we wanted to simplify the requirement, although it existed in the previous standard, we made a change to say, count to six and you're out, right? So we put a cap on the number of consecutive audits that an auditor can assess an organization, right? So that was a key change as well. In addition, we address some of the accreditation body requirements and made sure that they made sense, they added value. 

 Tim: One example that is the file review requirements. You know, when you're doing assessments, we looked at that and said, You know what, after you look at nine files is it really need to look at 14. You're not really going to gain value by doing four more. So we we adjusted that. And then probably the the last significant change is I learned a long time ago in my career that if you're going to add requirements, you also need to look at those organizations that step up and go above and beyond and perform. So we not only made a change to say that when you're performing poorly, right, your risk is high, that there's a need to add audit duration, but there should be a reward if you're a high performing organization and you're low risk, right? So that would reduce your audit duration. 

 Tim: And the last key changes that when you have an advanced quality management system, you use tools like the aerospace improvement maturity model to to get above just the barely met, if you will, to the more exceeds type category. We have a performance based surveillance and recertification process, another acronym, because we live in a world of acronyms, called PBSRP, and we're really proud of of putting that together and actually getting approval from the international accreditation forum of that process before we published it. But there's the carrot, if you will, high performing QMS organizations that step up and establish those controls and can demonstrate that get a significant reward by having the eligibility to enter a PBSRP process. All key changes.

 Susan: So risk and reward really comes into play. And you see that a lot. Darrell OCAP, obviously, big change in here, and the introduction of that, it's tell us a little bit about that. Tell us that the process that comes into play with that, the concept behind it, how does it work? 

 Darrell: So one of the items that the writing team became aware of immediately was fact that when you think about auditing, you think about how we audit today, I think about a certification auditor. What happens? They come in on a Monday, they audit all week. They go home on a Friday, on Monday morning, they're onto a new site. They're working all week, and they're auditing. And when you think about good audit internal or certification body auditing preparation is critical preparation to know exactly a little bit more about what you're going to so you focus in the right areas. And we this whole concept, which ISO, began to better clarify what is the difference between audit duration and audit time. Already talked about that. Audit duration is the opening closed to closing meeting on audit time includes that prep work and that that additional time to do the proper reporting at the end. 

 Darrell: So we needed a tool to kind of promote that whole concept of get some information up front. Let's understand what the risks are for that auditor before they enter into that site. And that was part of the reason behind the tool. The tool itself actually has gone through probably four years of development. I would do a little bit of work, come back, and then we tweak it, then we get some changes like we tweak it so that that spreadsheet that's been up, there's actually probably over four years now, of constant changes, of algorithms and coding and things like that, and the one we're at today is very well done. I think you'll find that it's very user friendly at the front end, lot easier than than the tool was when I was wrapping it up. 

 Darrell: But recognize there were a lot of people, a lot of reviews, that that influenced that I can walk you through the tool. One of the first things we're asking right up front is what standards got flow to contractually page one. And the point is, for us as an auditor, that's good information to know. If I know I'm coming into a site and I know that your as 9102 got flowed to you contractually, I can check to make sure your first article process complies with that 9102 requirement. The the other side of this is we know as as as the industry. You take that list of aerospace, aviation, space and defense standards that we've written, and if you were to look at that list later on, go, nobody uses this one. 

 Darrell: Why are as an industry, we wasting our time with something that nobody's using it? Is there a better way to get it used? Are there other standards that they're using that are more applicable? Maybe we don't have to duplicate that effort. So that first page is not only about setting the auditor on the right path, but it's also helping the industry to make sure we're we are working on the right things. Then you move into that process. Then the next part of that is really the risk analysis. And once you determine some of your audit, audit plan, what does the site look like? Is it, you know? Is it multi site? Is a single site? Then you go from there. Once you've figured that that information out, you go into the risk assessment. 

 Darrell: And the risk assessment has several different categories. I won't go through them all, but one of them is complexity. We leveraged MD five. There's a table in MD five that talks about a complexity of an organization. So when you look at that particular chart, bottom, left hand side, you know, I don't have I might be a single site, very simple processes, I'm going to be at low risk compared to somebody who's on the other extreme, which is a multi site. They have design authority, they do everything. They have unique and special processes, and they are going to be high risk just because of the nature of the complexity of their organization. We talk a little bit about internal audit in there. We've recognized that the internal audit program is a control point, and as a control point, if you've got a robust internal audit program, one that goes above and beyond. It's it's not risk based, it's you do a full QMS audit every year.

 Darrell:  You understand the numbers. You do additional auditing when that's required. You move your way up into that low risk category. There's a whole section in there about organizational risks. So this is where the organization can actually define what does it mean for on time delivery for that site, and let them make that decision. They can look at their day as an auditor. You can come along and look at it and go, okay, so you said on time delivery is 80% yet all your customers are saying 90% or better. That's an audit path to go down, but they can define their on time delivery, and they have to meet what the customer requirements are. What is their system for measuring customer satisfaction? What is their system for measuring quality performance. 

 Darrell: Those items are in the standard. We're not adding anything new, but we've incorporated this into the risk analysis tool. And then the last one is really looking at the past, you know, past performance on certification body audits. So if your pairs, if you look at the we've kind of leveraged the risk cube that's in those pairs. And we're saying if you are scoring a one which is very bad, you probably got a major and you get a non existent system that is like the weakest link, and that feeds into the risk tool that's a little bit about. And then you get into audit duration. 

 Tim: I really like the way you summarize that, and I think it's key, and it's very important that we understand that we didn't add any new requirements to the quality management system, but we wanted to take full advantage of those processes which generate metrics right that can be reflective of the QMS performance, and use that to establish a more robust internal audit plan mentioned customer satisfaction, right? So I think that's key as well, where we as stakeholders that utilize the certification process get that snapshot in time of how well our suppliers are performing. So all that's key to successful implementation. 

 Susan: It's perfectly okay. I like the fact that what seems like is that we really are able to zone in and deliver a robust outcome based on some key features that are individual to each site, and be able to address those and that preparation to have a more seamless outcome. 

 Darrell: Yeah, and you're right. It's all about the site, and it's all about the risk of the site. And when you think about what we're doing here, one thing I didn't say is, if you're a high risk site, you're going to get an automatic bump in the direction to the certification body is a minimum of a 10% increase because you're at high risk. And think about that. I'm not meeting my on time delivery. I've got poor quality performance. My customers are not happy with me as a certification body. 

 Darrell: They need to up their time to get in there and understand why and what's wrong, and are there non compliances and what, what portions of as 9100 are ineffective. So it's additional time on the other side, though, if I'm low risk, I actually get a benefit. I get 10% reduction over and above everything else off of my audit duration, because I'm meeting all my requirements, and I I've got a high performing system, not even talking about PBSRP. That's just our normal process. And just as you can benefit by getting a 10% reduction. 

 Tim: Hey, the other thing it stresses, I think we we need to add, is that involvement by the certified organization. Darrell will attest in the past. You know that scenario you said, where the auditor audits Monday through Friday, then he's off to the next client, auditing and auditing, well, what that drove was the standard boiler plate audit plans, if you will. But, but back to the organization, as Darrell described, it's key that the organization be involved in that planning process, right? They have to submit the OCAP data to their certification bodies 90 days in advance of the audit gives the certification body a chance to evaluate that data, but then, through that involvement, take ownership of it, right? 

 Tim: This is not an effort to grab additional revenue by increasing audit duration, right? It's increasing audit duration when it's justified, right? It's not just increasing because I added an extra standard to the matrix. So I encourage all those certified organizations out there, please work with your certification body, understand the process of how they evaluate that OCAP tool, get familiar with it so that as you're submitting that data up front, you understand, going into the process, how well your organization's performing, and then stand behind it and challenge your certification body when they add audit duration and level, set the expectation. When risks are low, you need that reward, and they should be reducing it. 

 Tim: So it's going to be key that organizations begin to become more actively involved in planning for a successful audit. So how do they become actively involved? I mean, are there, is there training available? What's the transition and the deployment? Can you talk about that time frame and that roadmap? Yeah, yeah. So we do have a comprehensive roadmap for transition, okay, it starts by getting all the committees transition to adopt the standards. And I'm pleased to announce that that's already happened, right? So the structures around the globe have already transitioned their systems and processes ready to support the accreditation bodies. 

 Tim: We have a number of accreditation bodies that have already transitioned, so they're already working with the certification bodies, and eventually the certification bodies will cascade that down. And one of the key things is having trained and competent auditors to be able to utilize the process. So we just recently rolled out the English version of the aerospace auditor transition training, the Delta training. So all you auditors out there, sharpen your pencils, and look forward to taking this excellent training. Then eventually we gotta start auditing to the new criteria. So that comes after the certification body has demonstrated they have the competent personnel, and the scope of their accreditation is updated to include the latest standard. 

 Tim: You should start seeing some of that activity later this year, and certainly, as we move into 2024 I will be the first to test that we are a very complex organizations with, I'll get the numbers wrong, but I think we have seven different languages that The IAQG operates in. So you can imagine all the translation activities that have to occur to get that auditor training material out there. 

 Tim: So some of the transition is based on that language being done. But ultimately, to answer your question about, how do organizations get involved? Well, guess what? They listen to Darrell, right? They listen to these podcasts. They go out to the website, to our requirements information. We're going to keep communicating that. We'll update it. We'll give you the latest and that's how we intend to get organizations aware of these requirements through activities like we're doing today.

 Susan: Wonderful. Thank you. I think at the moment, that's where I am with with my questions, are there any aspects, any features, any items that we didn't discuss about the changes brought by the 9104-1, or the the OCAP tool that we didn't cover? We want to address today? 

 Darrell: I would there are two things in there I didn't touch on. If you're familiar with the old system, the only chance of reduction was if you were less design. We changed that. We actually you could have a design house that does no manufacturing. You can now take a reduction for the fact that you're not doing manufacturing here, there. This is new. Purchasing used to have to be done in every single audit. But we've also recognized some of these multi site. They don't have purchasing, they might be centralized. So it changed the whole set of rules around that. So the tool is set up. 

 Darrell: We went back to one table, and we take reductions from a standard, and so you can take a reduction. I don't do purchasing here, take a reduction for it. You couldn't do that before. I don't do manufacturing. You can take a reduction for that. So we've broken that out. That's new. The other thing I'll just add is we've added a piece in the back talking about time reallocation, and we've given them the opportunity. Sit there and say, I have a high risk site or and I have a low risk site, I might look at a low risk site and saying, e are wasting our time in that particular low low risk site, and I want to actually spend more time in my high risk areas. 

 Darrell: We can reallocate time, provided you don't break the rules of MD five, and the table is set up to manage just that. You can't go below the time. You have to do the time, but you can reallocate it to sites that are, are of greater risk that you want to take a good look at those. Those are two things we didn't talk about. They are new. They're built into the tool. Good way to go. 

 Tim: I do have the opportunity to do a lot of training internally and as well as some external workshops, things like that. One of the things I always talk about is resources. Right? Where do I go to get information? So I'd like to give a shout out to you Susan and your team has done a great job of updating the IAQG websites and making sure that I've got a resource to go get information. So please all of our stakeholders out there, especially the certified organizations that, let's face it, only think about certification once a year, right? Where do I go to get the information? Please bookmark the IAQG homepage, and across the menu bar at the top, there's a there's an icon called certification. So I'd encourage all of you to go click on that, and you'll see a wealth of information, latest communication, and that's a resource for you, and I just would be remiss if I didn't mention that during this podcast. 

 Susan: Well, thank you, Tim. And again, you're taking words out of my mouth, perfect segue to make sure that everyone pays attention to the IAQG.org website. IAQG is also on LinkedIn. If you are on that social media, do follow us and obviously pay attention to all things on our podcast and the quality horizon. So again, both of you, thank you so much for your time, Tim and Darrell , this has been informative. Very much appreciative of your time discussing all of these changes and the OCAP tool. Best of luck with this transition and all the things that are going to be happening throughout the year. With regards to this. This is Susan Matson, and you've been listening to the IAQG quality horizon until next time. Stay safe all.