Implant brand portfolios, the potential for camouflage of data, and the role of the Orthopaedic Data Evaluation Panel in total knee arthroplasty

Listen to Andrew Duckworth,  Jonathan Phillips, Keith Tucker and Prof Haddad discuss the paper 'Implant brand portfolios, the potential for camouflage of data, and the role of the Orthopaedic Data Evaluation Panel in total knee arthroplasty' published in the October 2021 issue of The Bone & Joint Journal.

Click here to read the paper.

Transcript

Copy Transcript Copied

[00:00:00] Welcome everyone to our BJJ Podcast for the month of October. I am Andrew Duckworth and a warm welcome from your team here at The Bone & Joint Journal. As always, we would like to start by thanking all of you for your continued comments and support as well as a big thanks to our many authors and colleagues who've taken part so far this year. We hope that you're continuing to enjoy our podcasts and all our knowledge translation work delivered so far. This has included focusing on papers published each month here at the BJJ, as well as our special edition podcast series. The first of these is the insights from the US with our first podcast a great discussion with Professor Heather Vallier focusing on trauma. And last month we talked to the excellent Dr. Matt Abdul from the Mayo Clinic. 

Our other special edition series has been with our invaluable specialty editors here at the journal, which we hope is giving you an insight into all the vital work they do here, as well as providing an overview of the literature in that area.

So moving on to our paper for this month, firstly I have the pleasure of welcoming back and being joined again by our Editor-in-Chief at the BJJ, Professor Fares Haddad. Great to have you with us today. 

Thanks Andrew. Great to be with you.

 [00:01:00] Joining Prof and I today are the authors of an excellent study that's been published in the October edition of the BJJ and is entitled Implant brand portfolios, the potential for camouflage of data and the role of the orthopedic data evaluation panel in total knee arthroplasty.

So firstly, I'd like to introduce Mr. John Phillips, who is a consultant orthopedic surgeon at the Exeter Knee Reconstruction unit and a member of the ODEP group. Welcome John. It's great to have you with us today. 

Thanks for the invite.

And joining John is his cohort from the paper, Mr. Keith Tucker, who is the chair of the orthopaedic data evaluation panel and beyond compliance advisory group. Thanks so much for joining us, Keith, it's also great to have you with us today.

Thank you. 

So John and Keith as you state in your paper we all know that knee replacement is a highly effective treatment for arthritis and a range of other disorders around the knee. And there is a wide range of implant brands and joint replacements available and sort of related to this and potentially some prominent and publicized failures of implants in the past joint replacement surgery we know, and quite rightly is highly regulated in the UK through [00:02:00] a national registry. You also comment that the options available within many brand portfolios has grown exponentially over the past few years. And so sort of drawing all that together the aim of your study was to investigate the effect of the expansion of implant brand portfolios and where there may be a potential lack of transparency around a brand name. And you also aim to establish the potential numbers of compatible impact construct combinations that exist.

So Keith, if I could start with you as an introduction to the study, can you give our listeners a brief overview of the role of the NJR and how this content can compare with ODEP in this context, and also a bit about ODEP ratings and assessment? 

Well, thank you. Okay. Well, odep and the NJR were set up round about 2002. In fact, we're next year is our 20th birthday, I think. And that was in the wake of the 3M capital hip problems. Now a registry, as it turned out to be NJR was something many of us have been asking for for, for years. And we were delighted, it was going [00:03:00] to come, but it was never going to be a quick fix. So that's why ODEP was created by NICE with a requirement that manufacturers of hip implants should submit data to support the use of their implants. In fact, at that time there was very little data around except for the Scandinavian registry data. There was no registry data that as we know today.

And NICE they, they, they started off by telling us that we should have two ratings - ten rating for products, with data for 10 years and a three rating for new products. And anyhow, we grappled with this a bit immediately realized we've evolved. We've evolved. We've evolved. We realized that this was never going to work, was never going to be adequate. And so we introduced the five and seven year benchmarks.

And also the concept that once an implant was enrolled [00:04:00] in the process, manufacturers had to keep climbing up through the benchmarks as time went by. So nothing could be stationary. And we've now got benchmarks at 13 and 15 years.

That was for hips. NICE said that there was no demand for ODEP for knees. There hadn't been any complaints, nothing like a disaster to get things going on. 

But in 2012 BASK came to us - British association of surgery for knee to initiate ODEP for knees. And then shoulders came along in 2017 and we're now moving ahead with spine ankle, wrist, and hand. 

The one thing I think we do, like, I think it's reasonable for us to claim is that we have encouraged, strongly encouraged, manufacturers to look at data about their implants and by default it is now difficult for manufacturers to market their implants, certainly in [00:05:00] this country without, without an ODEP rating. 

At the beginning, as I say we use data from anywhere we could get it, and that was often Scandinavian registries. And we continued to use data from all around the world for ODEP benchmarks. And I should add that our ODEP ratings are actually used all around the world.

Yeah.

And we do that with, as I'm sure you're aware, we're all unpaid and we're proud of our independence. 

So that was a little bit about ODEP. With regard to the NJR as I say it was realized that a registry was an investment for the longer term. And nowadays of course, the NJR has a wide, wide ranging responsibilities for monitoring surgeons, hospitals, implants. And on top of that, all the research that goes along with it. But it mustn't be forgotten, [00:06:00] just one thing, that the original remit for the NJR was to keep records of patients with implants so that if there was ever a problem, we could go back. 3m we didn't know who had got the darn things. And so it was actually difficult to get back, get them back to the clinics. And when we had the metal on metal problem NJR was very quick at getting patients identified and back to their hospitals for checkups. 

So now we've got the national joint registry in our country. It's the largest registry in the world. And almost certainly the one most quoted and used - certainly by manufacturers because we have the supplier feedback and I think we are fantastically lucky to have this registry. But what we've got is big data, big, big data. And I think we've now realized that there is the potential for camouflage within big [00:07:00] data. And that's what we're trying to address. Thank you. 

That's great, fantastic overview of how ODEP has sort of evolved over time and how it has developed in the first place. And I think John, that sort of brings us very nicely onto sort of, you know, in your introduction, the study you're describing about the vast range of options that can now exist within a certain brand. So can you for our listeners sort of maybe expand on that and what potential issues that can give us with regards to what he's been saying about ODEP and NJR.

Yeah. Thanks. Great introduction actually. I think one of the main points I just wants to add to Keith was that when hips were introduced it was introduced as a femoral components, then acetabular components, and each got a rating. When knees were introduced a construct was brought together. So it was held that a tibial base plate couldn't receive a single rating in the same way as the femoral competitor couldnt receive a rating. So all [00:08:00] lumped together, but it was lumped together at a very granular level. So you had to have a certain base plate with a certain patella and a certain femoral component with a certain insert.

So if you, if you basically consider a basic knee replacement system, you have a base plate. You have you have a femoral component, which is probably going to be either in post-injury stabilized or pressure retaining a PS or a CR, and then the corresponding inserts, and that is with, or without a patella. So basically you end up with almost like a grid of four different options. 

So that would be a very basic system, but if only life were that simple because as you probably, as everyone knows, who's performed the replacement surgery, there are variants available within certain brands. And just to sort of expand on that, the options potentially available you can have uncemented versions, you can have a modular tibial base plate with a stem. You can have a mono block where the insert is actually attached to the base plate. You could have an uncemented [00:09:00] version. You could have an allergy type, you know, with alternative bearing, you could have a mobile bearing base plate. You could have fixed bearing base plate. You could have, again, a different material, cobalt chrome titanium, and you can have different shapes kiehls.

And so therefore I've just sort of listed eight potential different base plates that you can have all of which are pretty much compatible with the other systems. And then when you consider that you could have equal number of this different types of femoral components, inserts, different makes, different materials, different shapes sizes. So therefore that, that led us to sort of come up with this idea that, you know, how many potential combinations could you have within a system. 

That's great. I think that leads us very nicely to the paper in terms of what you discussed there, the number of combinations is just, we all know about it, but it's actually just mind boggling when you put that all together, which we'll come onto in your paper, but just before we do, Prof, if I could come to you, what are you your thoughts on these potential [00:10:00] issues? You know, before this paper has been published in the journal, but, you know, and, and maybe how we've dealt with these or not dealt with these in the past. 

Thanks, Andrew. I think this is a great paper because it articulates something we'd been thinking about, worrying about and talking about in the journal for a long time. And, you know, as you know, we've been pushing extremely hard for registries to clean up and improve their reporting. You know, we recognize the immense value of these registries, but you know, as early as 2013, 2014, we put out a couple of editorial, one on, you know, the trouble as well as the benefits of big data and another on how it should be interpreted with Dan Perry, Matt Costa, Nick Parsons, and that, you know, that brought out lumping and all those issues related to that. And I think this is just another reminder that we can generate hypotheses from big data, but then we need to take a step back and drill down. It's not to do down or denigrate, big data. Big data's really important. [00:11:00] It's an important part of what we do. And for any question in orthopaedics there's a number of ways of looking at it and we've always got to choose the best one. I think whenever we're looking at these things, it's very, very important that we bear in mind that the devil is going to be in the detail. So I really welcome this sort of concept of camouflage now being in the public eye and the great work this paper's done from that perspective, because it'll put the focus on these bigger data sets, moving forward, whether they're registries or quite frankly, some of the coding databases, which will suffer from the same problems.

Absolutely. Absolutely. So John, if we come on onto that sort of leads in nicely, sort of the, sort of look at the methods and the results of how you did this, and, you know, your methods for this study were based around creating hypothetical implant brands and the impact of implant variance within these brands. So can you give just a very brief overview of how your assessment of this was performed to answer the study aims, and then we'll move on [00:12:00] to what you actually found. 

I mean to keep it really simple it's very obvious why you, while we use hypothetical rounds. But basically we just did the maths and put things together and very simply worked out what happens if you added second femoral components and added second tibial components, and then very simply just did the math to it. And then just got somebody cleverer than me to check my maths. Fortunately, the maths was okay. But no, we kept it very simple. 

Yeah. No, absolutely. And I think that, like you say, the study, isn't not evaluating any specific implant or implant brand data. It's just hypothetical, like you say. So in your cause your study sort of highlights that there are sort of 30 commonly used mainstream brands of knee replacement in the UK. And the NJR reports sort of further subdivides these major brand names into about 49 groups for analysis. And you break the results down to sort of demonstrate the effect of addition of alternative implant options within a brand, a hypothetical brand portfolio. So John, if we sort of take the [00:13:00] first example that looks at CR and PS options, what does that sort of breakdown into? 

Well, simply that breaks out into that simple grid or you have a CR and a PS option, with or without the patella and the tibia is the same, for instance, you end up with four different options in a very, very simple. So, therefore if you have 30 brands then potentially you've got four different options with each brand. And that is if it was kept very simple. 

Yeah. And that's the sort of starting point isnt it, but then if you, when you move on to your next examples, you looked at the addition of sort of one to three various additional options. And so what effect did that sort of have in terms of the numbers? 

Yes. That's where it all started becoming interesting and basically for every additional, so for every second tibia option, maybe you had a modular tibia then that would have the effect of doubling the number of options. So you go from four to eight and then you had a second type of insert, maybe a cross-linked poly, that again doubles it to [00:14:00] second femur. So that takes you to 16. And then you had another femur and, and then basically you ended up with 32 different options. So basically every time you add a second option, with or without a patella, you end up doubling it to 32. And then by the time you start adding a third option, it doesn't double it. And actually the numbers slightly decrease the other third femoral components. You add it by sort of one and a half and then it, and it goes down to 1.3, if you have a fourth different options, but again, that takes us into the, you know, the, the situation where you could having three different femurs and two different femurs and patellars and inserts, you end up with the 42 or 48 different options. 

Then the interesting bit is what happens when you add in the uncemented options, because uncemented options really make the thing, the numbers go crazy because you have the option then of hybridizing them all. So you could have an uncemented [00:15:00] femur with a cemented femur. And so basically if you have your standard hypothetical group of four. And then you add an uncemented options, you quadruple the numbers. So you go from four to 16 different various options with uncemented, with all the various options. 

So if you have your numbers where you have two different versions of everything, and then you add an uncemented option have get to 128 compatible variance. And if you add an uncemented patellar onto that, then you go to 192 different options and it gets crazier and crazier. And we read in our paper, we put together a hypothetical brand with essentially three different options in each of them. And we came up with 750 compatible brands. 

And on the back of my fag packet, I did some work yesterday and I worked out that, I think if you use, if you have six, because there are some brands out there with six different tibias and six different femurs, six different patellas, [00:16:00] and there are over 10,000 difference compatible options that a surgeon could potentially use. So the numbers become quite staggering and quite interesting.

I completely agree. I mean, that's what struck me when I read this. Even if you look at the, the table five in your paper, just, you know, going down the femoral components, tibial component, inserts and patella, and not that many options, when you think about it, a great deal of thing, but then you're ending up with a total number of compatible construct complaint, combination to 768. It's, it's something that you, you, you, you think you probably should know, but it's, it's actually seeing that number print is quite, is quite remarkable. And so that's sort of, sort of, you know, from that though, what do you feel are, I suppose, the extent of that study without question and what it raises in terms of implications and questions moving forward, what do you feel though are the sort of key take home messages with that? And maybe any caveats that go with?

So, so my, my key take home messages are that [00:17:00] surgeons should hopefully be aware now that there are multiple different variations within a brand that has been labeled a good brand potentially, but there are multiple different variations. So if you start using more niche options within a brand surgeon's must be aware that there may not be data available to support the use of that particular construct. 

And then also surgeons should also be aware that through this sort of big data sets a lot of these niche combinations may well be camouflaged and then the results may be camouflaged within the larger datasets within a brand. So there may be a slightly odd combination of now what we do know through, through working with ODEP and through looking at the NJR research is that not all knees are the same and not all hips are the same and not all implants are the same. So some implants may be [00:18:00] performing better than others and some implants may be worse than others. But if you perform a niche implants, which hasn't got many numbers, you won't know the results and they might have terrible results. You never know whether a certain niche combination that maybe has only been performed four times ever that all four of them have failed. We just don't know that. But in those four, within the 10 thousands in the bigger data set in the NJR, those results would be camouflaged. So what we've highlighted is that there's this potential for camouflage within data. And so the only way really, I feel as though surgeons potentially can protect themselves is by referring to ODEP. So ODEP independently analyzes constructs, which have been collected together. And there are many among the ODEP websites freely available. And then your surgeon can check that a certain implant with the femur they prefer to use and the patellar that they like to use. And the inserts and the tibia [00:19:00] have got independent verification at 5 7, 10, 15 years. I guess those are my sort of messages that just make surgeons open their eyes to the implants they are using. 

Absolutely. I think that sort of brings us nicely back to yourself Keith. So what do you feel the implications are building on that from John for both bodies, like ODEP and how surgeons sort of interpret those ratings?

Well, I completely support everything that John's just said. And I would just like to add at this stage, yes, this is a hypothetical study, but as chairman of ODEP, I can tell you that it fits pretty smartly with a lot of the knees that we look at, particularly with the big manufacturers. There's one manufacturer that has got knee constructs in ODEP. They've got 50 of them. When we're doing ODEP meetings, I mean, we will go through a very large number of constructs from, you know, just one manufacturer. We also know that [00:20:00] there are some constructs being used, which do not have an ODEP rating. We've done the drill down. We know that that's the case. We're not quite sure how, you know, how many there are of those. So when you say what about the surgeons? Well I think that the surgeons must make sure, just as John said, that the construct they're using, they should be aware of the data that's about it. And ODEP is somewhere they can look.

And you said, and true that, you know, what about the question of small series and John was just referring to, to, you know, maybe, maybe one construct has only been used four or five times. And then if the statisticians will say, well, that's not statistically significant, you can't draw any conclusions from that.

Well, I disagree to a degree because if a surgeon finds that only a very limited number of a particular construct was being used - [00:21:00] four or five, isn't that a message in its own right. You know, you've got to question, if you're going to do something, which is only be used four or five times by definition, there isn't a lot of data to support its ongoing use. And I think yes, small series have their major disadvantages, but I still think there's a message from a small series. 

Yeah, absolutely. I think that it raises a very, very important point, actually not to disregard, like you say, if there are only, only four have been done, it raises a question themselves, the surgeons sort of, I suppose, Prof, that brings me back to you. You, you know, in terms of your thoughts on the implications of this work for both surgeons and registers. And you're asking the same question I've asked Keith there as well, but how do you sort of think we move forward really in terms of, you know, not only our implant regulation, but also, you know, you know, management of innovation, you know, we do want to push things forward. But how do we sort of do that safely and, and, and use big data to do that the best way we can?

There are a couple of things, Andrew. I think there's some great messages here. [00:22:00] I think the other caveat to put in here is that beyond the implant constructs you've got here, you've also got different alignment philosophies, different alignment results, because what you end up with may not be what you aim to end up with and then different enhanced technologies. There's all sorts of other variables that are brought in. So when you really analyzing this stuff, you've got to drill into the detail and that's got big relevance, you know, we have had some reports in the journal this year, for example, of an implant which had a high failure rate in certain hands, but the defence is it does well in the registry. Now, if you follow that a little bit further forward, it may well mean there is a certain construct done in a certain way the implant manufacturers change that tray. So there must be a signal there that's been noticed internally and yet there are patients walking around with that tibial tray. So there are for our profession, there are implications for how we react to that and what we do. And we can't just hide behind that registry data. 

But moving forward, this is a really important thing in terms of when surgeons are going to [00:23:00] change practice they need to take a step back and consider, do I know how well I'm doing with my current constructs? Do I have my own data? Do I have my own outcomes? And how do those correspond with the outcomes out there both in the registries, but also in the peer-reviewed literature. And then if I'm going to change, how am I going to measure my own outcome? And if my numbers aren't going to be big enough, where am I going to get that external data? Because the overall registry curve is not really going to give them all the data they need. And that's a really key message from John and Keith that everybody needs to take away. This whole concept of camouflage means we need to break down into exactly what people are going to do and you know, this is what RCTs do beautifully. So we're going to go back from looking at big data for one valuable end of the scale, but actually looking at small mechanistic studies as a way of comparing A versus B still being a very valuable part of what we do in orthopaedics. So really I think the full gamut of [00:24:00] methodology is going to be. 

Yeah, no, absolutely Prof. I completely agree. And I think that's, what's so great about this paper. Like you say, it is based on hypothetical combinations, but you know, probably very real in many ways, but raises some really interesting questions for us moving forward, not only as the individual surgeon, but as interpreting the data as a community as well. 

Well, everyone, I'm afraid that's all we have time for today. But thank you so much to you all for taking time to join us and congratulations, John and Keith for a really excellent study that without doubt is an invaluable addition to the literature, and I'm sure will lead to much thought and discussion and debates that amongst our listeners and readers. And to our listeners, we do hope you've enjoyed joining us, and we encourage you to share your thoughts and comments through social media and a like. Feel free to post a tweet about anything we've discussed here today. And thanks again for joining us.