On Expertise

A slight twist on my normal Hacking Academia video intended audience: today's "On Expertise" video is targeted more at the myriad industry, government and other sector workers who are interested in getting expert advice and wonder how to find and assess it.

It's not a comprehensive guide but covers a few concepts that I've observed to be particularly important to consider in your expertise-seeking journey:

👉 what sort of expert do you need, and do you really need the world's leading expert (a relative claim) or do you just need an expert with a level of expertise sufficiently above your own to help you take that next step in understanding or strategizing?

👉 ELOQUENCE DOES NOT INDICATE EXPERTISE. But experts (for most types of expert interactions) should be good communicators, especially in written, verbal and visual forms. Anyone bright and quick at picking up things can, especially in this ChatGPT age, pick up a topic quickly and sound good in a 30 minute keynote talk or a 45 minute panel. 

👉 In fast moving / newish fields, you will encounter a lot of experts whose claim is based on expertise adjacency - they've switched from a different but related field / domain, and only picked up this particular expertise area relatively recently. A lot of the experiences from those other fields do usually transfer: the trick is to be aware for the gotchas: the concepts that definitively *do not* transfer and hence where they're starting fresh. Expertise adjacency is a necessary part of the game, because some of the new fields or domains have only been around for a few years rather than an entire career.

👉 Expertise in the aggregate can make up for (or be more cost effective) than sourcing the "uber expert in all things you need": sometimes you will still get benefits from the uber expert because they will connect concepts that the committee or panel in aggregate may not

👉 An expert can help you vet an expert: your pet expert doesn't need to be exactly in the same field to be useful in this manner

👉 A simple test of deep expertise for many areas is a multi-hour, intensive discussion (pref with your pet expert in tow). You will quickly tell the difference between superficial knowledge and genuine deep knowledge and experience. Superficial experts will run out of reliance on superficial catch phrases like "garbage in = garbage out" - these are helpful at entry level but not so much in any deep discussion.

👉 For the experts: aim to make yourself increasingly redundant: empower the people you're helping! It's really satisfying...

YouTube: https://youtu.be/pFkikdTAZoM

#HackingAcademia #expert #expertise

Transcript

SPEAKER_00 0:00

It's Friday afternoon, and I wanted to do a video touching on a topic that I've been thinking about doing a video on for a while, which is the notion of expertise. A term like expert or expertise gets thrown around quite frequently. As researchers, we write about how we are experts in a certain area. When you see a biography for a keynote speaker at a conference, they'll often talk about how they're an expert at X or Y. And I want to talk about a few key themes or concepts that come up with expertise. And this video is not so much targeted at the so-called experts themselves or researchers, but just as much at the end users. So people in industry or people in government who are looking for an expert for a variety of reasons, and I guess thinking about the ins and outs of how they go about finding one, especially how they go about assessing one, and then how they can get the most out of that expert for whatever their ultimate goal is. So let's do informal definitions first. I'm gonna have a somewhat narrow bespoke definition of uh expertise, but I would define it as someone who is deeply knowledgeable in a certain field or application domain, and my expertise is of course going to uh bias towards the more technical areas. So I work in areas like artificial intelligence, robotics, and autonomous vehicles, but also most importantly has the ability to then relate that deep knowledge and experience to the circumstances or the scenario of the third party which is looking to them for advice or insight. So it's not just the knowledge itself, expertise in my book would also be the ability to apply it in a deep and meaningful and accurate and highly relevant way to the particulars of the organization or individuals that expert is talking to. People will often talk about different levels of experts. So, sort of someone is a junior expert, someone is a world leading or world-class expert, and it's worth briefly thinking about how you describe the level of expertise of someone. And it's important to realize there's really two very different ways of defining it. It also relates to, I guess, how you assess learning outcomes whenever you're teaching students something. The first one is a relative one. So this is often phrased as this person is the world's leading expert in topic X. And that is inherently a competitive or relative indication of the expertise. You're saying they know more about this particular topic than anyone else in the world. Now, the significance of that statement is going to vary hugely depending on the nature of the topic or domain in which that expert is working. So if this is a topic like uh say chemistry or AI in general, these are massive fields with hundreds of thousands or millions of people working in it internationally. And so, if someone is legitimately or even somewhat legitimately got a claim to being the leading expert or one of the top 10 experts in the world, um, that is a relatively notable achievement just because it puts them in a very elite echelon amidst all of those other workers in the field. Does that mean they're going to be useful for you? Not necessarily, but it's still notable. It also doesn't mean the opposite. There are a lot of experts in very small, very narrow fields where perhaps there's only a handful of experts internationally. And that does not mean that they don't bring the same level of value and experience to that particular problem. It's just that they haven't necessarily had to show that they have value to offer over those millions of other experts around the world. So it's not that one or the other is better, but there is a difference. The second way to think about expertise is in an absolute sense. So does this expert have sufficient expertise to be able to do a pretty good job of helping me with the thing that I need help with? Now, this is a totally different way to measure expertise, it's a little less explicitly competitive. And for a lot of people, this is the type of way of thinking about expertise that you will need because you often don't need the world's leading expert. The world's leading expert is often very expensive or has a massive ego, sometimes, not always, and often you don't need that. So, for example, as Chat GPT revolutionized everything and industries, companies, and just about everyone from government was sort of desperately seeking to find out more about this whole AI thing, you didn't need necessarily the number one world expert. You needed someone who had solid experience and technical understanding of some of these technologies to give you that initial advice to help you understand the broad parameters of what this sort of technological transformation was all about, and then to relate it to how it might immediately or in the near future impact some of your core operational processes. And in that case, there would have been probably millions of people worldwide who could have done a very good job right out of the bat for you at that point in time. A key consideration is how you find an expert. There's lots of ways to do this. You can go to the organizations that typically have an oversupply of the experts. Sometimes this will be the university or research sector, sometimes this will be more so industry players, sometimes it will be from the government or the not-for-profit sector, and sometimes it will be more freelance experts. But there will be known organizations or memberships or associations that have a list of these experts. You'll also see experts or again proclaimed experts pop up at a right a large range of events, so giving public talks at conferences, especially in the domain that you're interested in, being on boards, advisory roles, providing expert witness capacity in a court case, and a number of other activities where you will tend to find experts in the particular field being discussed. A really, really important point to note is that eloquence does not equal expertise. One of the things all end users or people who are looking for experts should realize that just about anyone who is reasonably bright and a so-called quick study, so someone who can amalgamate and absorb a bunch of information very quickly and who is also very eloquent, can get up on stage at a conference or in a panel or sit in a 45-minute meeting with you and sound incredibly knowledgeable and polished about the topic that they're talking about, even if they've just spent one day previously cramming up uh with Chat GPT and reading a few papers or technical uh briefs on the subject. Eloquence is not a direct indication of expertise at all, and it can be very, very dangerous because a lot of people are very uh eloquent, very good communicators, and have worked hard to develop that skill set, but don't necessarily know anything about the topics that they're talking about. And especially for an audience who is not an expert in the topic, it can be incredibly difficult to distinguish between actual expertise and someone who is just eloquent and tell a nice story. As a side note, of course, you want your experts to be able to communicate well, especially in verbal and written form and perhaps visual form. If they can't communicate well, then their ability to actually dispense that expertise and for people to get benefit from it, of course, is going to be limited. So being at least somewhat eloquent and commute good at communication is a key capability for an expertise, but it does not by itself indicate that there is substantial expertise there. One of the phenomena you'll encounter with experts quite a bit is what I call expertise adjacency. So in fast-moving, especially technical fields, very few people have spent a 20 or 30 year career studying the narrow, very focused topic that you want help with, often because that topic has not been around for 20 or 30 years. And this is especially true in areas like parts of AI and robotics, but it would also be true in other fast-moving fields, so some aspects of biotech. And so what you'll get is what I call expert adjacency. And so these are people who have a lot of experience either in the broader general field, but have only picked up that very narrow specific technical area relatively recently, or people who have, for example, done a lot of work in a particular technical area but in a different application domain to the one that you're talking about. So in my field, it might be someone who has spent a heck of a lot of time working with drones, for example, and has relatively recently moved into the area of, say, autonomous vehicles or robotaxis. Expert adjacency is not inherently a problem and it's a practical thing. You're just not going to get enough access to expertise if you don't work with experts who have switched or or pivoted slightly in their area of expertise. And expertise in one related field often does transfer quite nicely to another. The key pitfall is that it doesn't transfer perfectly, and so there will often be blind spots, often unknown to both the expert and the people who are talking to them in terms of analogies or experience from one field that most definitively do not transfer to that other field or that other experience, and so it pays to keep an eye out for those gotchas. So, what are some of the hallmarks of what could potentially be a great expert? Well, there's a few key things to look out for. One is the expert will be very aware of where their area of expertise lies and where the boundaries to that expertise uh occur, and they will always be evaluating whether the particular question or topic that they're talking to or answering is within that core area of expertise or has strayed outside of that area of expertise, and they will flag or communicate to you that that is not an area they're so confident in. They'll maybe attempt to assert that they have some experience but are by no means a genuine deep expert in that area. And then, if you want, they can speculate, noting those caveats, or you can cease the conversation there for that particular tangent, go find someone else to talk to that particular point, and then go back to the areas where the expert is much more confident in. People uh find this and I find this incredibly difficult to do because if you're in the middle of a long and detailed and interesting discussion, uh you get into a flow state, and it can be very hard to detect when you've actually moved outside of your core expertise zone, but it's something that you should be very much aware of. Another key way to uh vet or evaluate an expert is to have another expert help you do this. And the expert, although ideally is in the field that you're trying to find more expertise for, it doesn't actually matter that much. An expert in any broadly related uh endeavor will be very helpful in vetting the expertise and true knowledge and capability of an expert that you're looking to hire or take on. They don't have to have exactly the same skill set. Obviously, you want to find someone who is fair and constructive and objective, not someone who has an axe to grind because maybe you didn't give them the contract, but they will be very, very useful in helping you with your vetting process. A third primitive but remarkably effective test for expertise is to simply spend a long sustained period of continuous time talking to that expert about the topic that you need help with. As I mentioned before in this video, it's very easy for a bright, eloquent person to cram up on a topic and get a superficial or even moderate level of understanding of a field. And sometimes that's enough in some areas of teaching and learning or base level advising. Uh, really, you just expect that the person advising you has a has a meaningfully higher level of knowledge, but they're not necessarily a deep, deep expert. But if you do need that deep, deep expertise, and I'll talk about that in a moment, um, just sit in a room, preferably with your own pet expert, and talk to that person for say three or four hours non-stop about that topic. If the person is not a genuine world-class deep expert in that area, they will find it very difficult to talk at a sophisticated, nuanced level about that topic. They'll be good at catchphrases, rules of thumb, uh, but they will start to falter once you get into the more nitty-gritty details. A classic example of this is looking at something like second degree, third degree, and fourth degree effects. So you broach a topic that you're discussing, you go, well, if this, then that, and what would happen then, and then if this and then that, what would happen there? And once you get several layers deep, um you can really genuinely tell whether that expert still has deep, insightful advice to give because they fundamentally know what they're talking about, or whether their expertise is somewhat more superficial in that area. So, how do you know how good and deep and rare an expert you need? Well, that is the billion or trillion dollar question. And I don't have a fantastic answer for you. What I would emphasize, and I'm going to contextualize this in the context of AI, is that at the entry level, most expertise is going to help you because you're getting to grips with the new technology or transformation, you're becoming more familiar with it. And at that point in time, phrases like AI is just like a calculator or garbage in equals garbage out are arguably reasonably helpful to getting you at that entry level of familiarity. One of the perverse things though, and I see this happen all of the time, is that people will hang on to those initially useful catchphrases or catchphrases or rules of thumb for way past their expiry date. To give you a concrete example, the garbage in equals garbage out saying is useful for indicating the broad concept that there to some degree you need reasonable data to train these systems. But to give an example from my own field, my own field of uh robotic mapping and navigation and probabilistic robotic mapping and navigation is entirely based on the fact that the data we feed into our systems is to some varying extent garbage, noisy, biased. And a large chunk of the field, including a lot of the AI work, has been specifically developed to ingest that garbage as well as some of the good data that exists amongst it and make the best use of that data whilst filtering out or downweighting the influence of that garbage. And so the garbage in equals garbage out and other sort of very superficial catchphrases should not really be used beyond that initial introductory period where the main win or objective is just to get some base level familiarization. This is an example where deeper level expertise is needed because only people who have genuine expertise in the area that I just mentioned are actually going to understand how those things work at a more technical level. The final important concept I want to address is expertise in an individual versus expertise in the aggregate. So there's two key dimensions here. First of all, it's very hard to find an Uber expert who is an expert across all the multifaceted aspects of what you want to do. In a more commercially oriented case, this would be an expert who has both the deep technical understanding but also a detailed understanding of the economics or business models of your particular sector. Often you'll then try to substitute for that very hard-to-find Uber expert by finding up a committee or a panel of experts who meet that expertise requirement in the aggregate. And that is a very sensible, very widely deployed strategy. If you have a really critical need though, and you have the resources, you do still want to try and find some of those experts who have that rare combination of deep expertise across the multiple areas that you're interested in, because they will be far more efficient and their throughput will be much higher than doing it via a committee. Of course, the committee has other aspects of benefits like diversity of thought and diversity of perspectives, which can be very powerful. But if you can find those Uber experts, they will be able to make breakthroughs quicker or more insightfully sometimes than you can do just with an aggregate committee-based approach. A final more philosophical note about expertise and advising, and this is a philosophy that I've sort of grown and developed over many years of doing it, is one of the really satisfying roles for an expert is to make themselves gradually more and more redundant to the topic being discussed. And so a great example of this, and I've been very proud of having played a tiny, tiny role in this, is working with people over many, many years and initially, understandably having them lean very heavily on your expertise, and then over the years being able to sit back more and more and just listen to them talk and realize that they've either matched or perhaps surpassed your own levels of expertise in the topic and taken it to a whole new level. That, although it can be a little bit confronting because you may feel not so much needed anymore, there's lots of other needs for expertise that are always out there. And it can be very fulfilling to have left a sort of empowering, self sufficient mark of expertise with the people who initially came to you for help.