Learning, education and training are undergoing major transformation in front of our eyes and not just because of the pandemic and the remote learning we are experiencing as a result. A vast amount of data generated by schools, learning environments, and training systems about the learner is now available. When you combine that data with advanced technologies such as artificial intelligence, data analytics, and adaptive learning systems, it not only helps us measure and understand how humans acquire new skills and absorb new knowledge, but as we will learn on this episode of MINDWORKS, it may also have the power to address societal problems of learning equity.
Learning, education and training are undergoing major transformation in front of our eyes and not just because of the pandemic and the remote learning we are experiencing as a result. A vast amount of data generated by schools, learning environments, and training systems about the learner is now available. When you combine that data with advanced technologies such as artificial intelligence, data analytics, and adaptive learning systems, it not only helps us measure and understand how humans acquire new skills and absorb new knowledge, but as we will learn on this episode of MINDWORKS, it may also have the power to address societal problems of learning equity.
Janet Spruill: The promise of adaptive learning covers a lot of ground. I think that one that's really important, especially in these times to talk about is the issue of learning equity and driving learning equity at scale. And what I mean by that is there's a fair amount that's published about the potential of adaptive learning technologies to drive equity at scale on higher education. But I think it goes more broadly than that. I come back to think about the personnel that we interact with, the young Navy recruit and how adaptive learning can help these young sailors who come from all backgrounds and all achievement levels. So whether it's in a college or a military technical school, these underserved students can feel perhaps disproportionately unprepared for many aspects of their learning experiences coming into them. I think adaptive learning can really serve almost as a great equalizer to help them succeed and build a lot of confidence as they go through their learning career.
Daniel Serfaty: Welcome to MINDWORKS. This is your host, Daniel Serfaty. This week, we are going to explore another topic on our quest to deepen our understanding of how human think, learn and work. And we're going to focus specifically on the learn part. Learning, education and training are undergoing major transformation in front of our eyes and not just because of the pandemic and the remote learning and education that we are experiencing with it. But the vast amount of data generated by schools, by learning environments about the learner is now available. And when you combine that with advanced technology such as artificial intelligence, data analytics, adaptive learning system, it does not only help us measure and understand how humans acquire new skills and absorb new knowledge, but as we learn, it also has a power to address societal problems of equity.
In light of this revolution, I'm delighted to have my two colleagues from Aptima join me today and take us on a journey of exploration, looking backwards and forwards into the world of learning, training, and education. And it is my pleasure to introduce Ms. Janet Spruill who's Aptima's Vice President of Programs and has more than 25 years of expertise as a human performance technologist and she developed and work with training systems and Dr. Krista Ratwani, who is a Senior Industrial Organizational Psychologist and expert in leader development and training, and is a Senior Director of Aptima's Learning and Training Systems division. Here we go. Welcome to MINDWORKS.
Welcome to MINDWORKS Krista and Janet. What made you choose this particular domain? The domain of understanding how people acquire skills, how people acquire mastery, how they develop both in the research side on Krista's side of the equation and on the implementation and the fielding and the execution side on the part of Janet. Let's start with your Krista. Why this as opposed to all the other career choices you could have made?
Krista Ratwani: In some ways I would say I kind of fell into it. I focused in grad school on leadership and leader development. And today the field of learning and training and education is changing. And a lot of the qualities that we are trying to develop in all individuals is really what I think about as key to leadership and great leadership. So things like critical thinking, things like adaptability, that has been my focus in terms of thinking primarily from a leader perspective. But now I think that that's getting broadened to this more just general learning area. And we want all individuals to have those types of skills. And so for me, it was that transition from focusing purely on it from a leadership perspective to more of the general learning perspective.
Daniel Serfaty: So in fact, it's the acquisition of critical skills and complex skills that is your research interest?
Krista Ratwani: Yes, exactly.
Daniel Serfaty: Janet, you came at it from a different angle. Tell us why this field? You could have chosen any domain you wanted.
Janet Spruill: I think like Krista, I came at it from a little bit of a different direction. I actually started my career in the field of human computer interface design, where I was working with organizations to redesign large corporate computer systems and doing what one of my customers at the time called designing for training avoidance. So looking at how we can redesign these systems to minimize the training that was needed. So through that, I've got exposed to many, many training classes that were teaching students to kind of click through a system calling it training because of course their systems weren't at all intuitive. So I got really hooked on applying my human computer interface experience to learning design, which was recently important at that time when training programs were moving online and beginning to incorporate more media and technologies.
Daniel Serfaty: I don't think the term learning design maybe familiar to all of our audience, what does it mean to design training?
Janet Spruill: So for me, I think that it covers a lot of aspects. So there's the instructional pedagogical design to ensure that the content and the learning strategies that we're selecting will result in the learning and performance outcomes that we expect. It also means blending the right delivery methods to be well aligned with the content that's being presented, whether it's a live training activity within a classroom or an immersive simulation or a job support aid or access to a short micro learning video.
Daniel Serfaty: I like that term a lot, the term learning design, because it's already introducing what we're going to explore in the next hour. And that has to do with the fact that learning is not something that happened. Training is not something you do but you have to design it the same way you design an engineering machine or a bridge, and it has some structure to it. It has some maintenance to it. It has some modeling associated with it. And I like that very much. We're going to explore that structural approach to the acquisition of skills and knowledge a little later. Our audience is saying, "Wow, we have two very intelligent guests with experience. I wonder what they do actually every day." Krista, what is it that you do?
Krista Ratwani: I asked myself that every day. There's a lot of variety, but from the perspective of what am I trying to do learning wise, and what am I hoping to do, it's really about developing those tools, those methods, those processes that will help people, that will help them learn more effectively. So get to the heart of what they need to know, allow them to enjoy the learning process. And in some ways also help them learn more effectively. We all know that organizations put a lot of money and resources into training activities, into learning activities for their employees. And so getting that return on investment is really important. And you want to look at that and help organizations do that, but you also want to ensure that the individual going through the learning experience is getting something out of it. And so that's really at a high level what I tried to do.
Daniel Serfaty: And I assume I'm not divulging any secrets here by telling our audience that you are the senior director of a major division. So you have several teams working for you on different projects. And so can you give us an example of what you just described for the audience to understand what does a senior researcher in this field do for a living?
Krista Ratwani: So the projects certainly vary, but one thing that we've been working on for a couple of years that I think has many different components to it is with the army and with the fairly new role of the military advisors. These are individuals that are getting deployed to go help advise counterparts in other countries. And so not only do these advisers mean to have all the tactical and technical skills that you would expect any soldier to have, they need to know how to shoot, move and communicate, but they're expected to build trust and build rapport with those individuals that they are now interacting with in a different country. And so what we've been brought on to do from the training and learning perspective is to really help infuse those softer skills into the curriculum. And so we're doing that in a couple of different ways.
One is to help the instructors that assess how those skills are being developed in the students. So if the students are actually developing those skills and then the compliment to the assessment is how can the instructors then provide the developmental feedback that those students need. Because feedback is a huge part of being able to learn really well and learn effectively. So it's developing those mechanisms that will allow somebody to learn. And then the other part of it is really helping the instructors set the scene so those types of skills can be developed. So in some sense, it's about developing little snippets of scenarios that can be injected into a preexisting curriculum. So enhancing the existing curriculum that the army has developed already and having them add additional scenarios and additional things that they can put into the curriculum to really drive home those learning objectives and those skills that they believe these advisors need to continue to develop.
Daniel Serfaty: You are not just giving advice to the US army, you also are doing what Janet mentioned earlier, training design in a sense. You're designing the training or at least some enhancing of the training that they're already doing, kind of improving through science what they are doing. But you said a word, you said soft skills earlier, I'm not going to let you get away with it. I'll come back a little later to ask you what do you mean about soft skills. But Janet, I have the same question for you. You are the vice-president for programs and you are managing many programs and many things, what do you do when you go to the office?
Janet Spruill: Well, surely I have a great job. I work with a really incredible team and we help organizations to improve their large scale learning programs. And we do it through a smart introduction of technologies and new methods for learning. We get to work closely with a number of government thought partners who really have a passion to improve their training programs and provide the very best training they can to help their personnel be mission ready. Because a lot of our customers are in the department of defense, a lot of their missions are pretty high stakes. So we take that really seriously. So for example, this year we've been working closely with one of the US Navy school houses, the Center for Surface Combat Systems who has quite a progressive approach to modernize training. They were an organization who was an early adopter of simulation-based training and fully immersive, and more recently augmented reality and virtual reality. And we're now working with them to insert some significant efficiencies and modernization into some of their training programs through some adaptive learning capabilities that I know we'll get into a little bit later.
Daniel Serfaty: And both you and Krista mentioned basically that you're working with agencies or units within the Department of Defense, soldiers and sailors, in a sense for the implicit assumption here and I don't want to sound overly dramatic, but the implicit assumption is that good training will save lives. You mentioned the term mission critical, and I think it is important. A lot of the domains that you are supporting that your teams are engineering technologies and injecting scientific principles into our domains in which lives are at stake, whether it's in the military domain or the law enforcement domain or the healthcare domain, that is an important part of why perhaps training and the deep understanding of how learning happens is important. Because at the end of the day, it's not just about the acquisition of skills, it's also about saving lives. Do you agree?
Krista Ratwani: I absolutely agree Daniel. I think that we certainly focus on saving individual lives. Many of these missions relate directly to national security and global security. And so in an example, we know that the military organizations have focused a lot this year on pushing through and streamlining the acquisition process for new equipment, new systems, including new weapon and defense systems. If they push those out very quickly but they can't also accelerate the time to get personnel trained, then we'll have equipment that can't be fielded or is fielded without the personnel to get really operated. And that can become a real national defense issue as well as an individual safety risk.
Daniel Serfaty: That's a very good point. Thank you. I'm going to ask you to go back into your own personal memories for a second. Leave the Department of Defense, we'll get back to it in a few minutes that we explore more in detail the technologies and the enablers of great training and great learning. But all of us have had in our past learned since kindergarten even before, we've learned in our colleges and universities, we are learning during our professionalized. Before we jump into the technologies, learning and training happens not just through technologies or enabled by technologies, but because of great teachers and great trainers.
Have you had a teacher in your past that really changed the way you look at the world? I know I had but you are my guest today, so I want your stories. Krista, can you share something about not only who was the teacher, but why do you think that teacher changed things in your way of learning and your way of looking at the world? Then later, I want you to speculate of whether those great teachers, the skill that they have could be somewhat replicated or mimicked through advanced technology. So let's start with your stories first.
Krista Ratwani: So the obvious answer that I have to give is my mother because she was my eighth grade English teacher, but I will leave that aside. I've had a number of great teachers that I can remember all the way from early education through college and grad school that I just look back and I think, wow, they really cared. And they were so passionate. One teacher in particular I can remember was from high school and she just loved what she did. And it came across and every single interaction with students and just in the classroom, she was the type of teacher who loved to keep you engaged. She would dress up for different days. This was an English teacher as well. And she would come in when we were reading Macbeth in her witch's costume or whatever it was, and just to really immerse the students in the material.
And I think that that's just key for me when it comes to learning and training. As I said, engaging that learner is so key because if you can't help them, they're not going to retain the material. I mean, there's obviously other variables at play there, but in some ways the relationship is that simple. And so she really just made an impact in terms of how she taught and just again, seeing that passion.
Daniel Serfaty: What a great example to say things like caring and loving and immersive and engaging? What a challenge for our engineers and our technology developers to mimic that, isn't it? Janet, tell us your story.
Janet Spruill: So I absolutely plus one everything that Krista said, especially about having energy and showing your love of the topic. So the teacher that comes to mind for me, the examples, actually, two teachers that I will compare and contrast. And it was actually a music teacher. When I decided to play a traditional stringed instrument called the hammered dulcimer in my early 20s, my first private lessons were a disaster because the teacher started with hours and hours of music and chord theory, she totally lost me. And I almost gave up playing, but a friend of mine convinced me to try another teacher. I did that. And the very first time I met with this new teacher, she had me playing a simple song in our first lesson and I felt successful and I felt excited that I could do this. So that experience really stayed with me and then impressed on me the need to help make students successful and to engage them as early as possible in the learning process.
Daniel Serfaty: This is a great example too because it also illustrate really one of the key themes I would like to move and explore with you when you compare those two teachers, both of them were teaching. The music teacher was actually teaching solfege and in music theory, but no learning was happening on the receiving end. So my first question is really this notion we talk about that in one sentence training and learning, training and learning as if they were synonymous words, but they are not. Are they? Krista, is there a difference between training and learning?
Krista Ratwani: I think so. Some people may say that it is just semantics, but at least to me when I think of training, I think of those more formal type of experiences. I'm going to go take this training course, or I'm going to go take this class. And I think unfortunately the reality is just because you are going to be trained does not mean you actually are learning anything. You can sit all day and listen to somebody talk or read a book or whatever the mechanism is that you are getting trained by, but that doesn't necessarily mean that you are fundamentally learning new skills or gaining new knowledge. To me, that's what learning is. There is some actual change in what you know or what you can do. I mean, I'm not a neuroscientist, but I think that there's an actual changing your brain that happens when you learn something. And I don't think that that's necessarily true when you just talk about training itself as an event or as an activity.
Janet Spruill: To build on what Krista was saying, in its simplest form, I think about training as an activity and learning as an outcome. And so in thinking about learning, what's most important is related to transfer of learning, having a set of learning transfer, which is a process of learning, which gives you the ability to extend what's been learned in one context to new contexts, which is critical. Training on the other hand is more about completion, learning I believe is more about transfer.
Daniel Serfaty: Oh, that's interesting. Can you think of an example of this notion of transfer learning in something, a skill or gain knowledge in one domain, but being able to then transfer it and apply it to another domain?
Janet Spruill: Absolutely. So I think that it can apply in skills-based training or in soft skills training. So think in the soft skills arena of negotiation skills. You can take a course to learn how to negotiate a business deal and really learn the fine art of that and the workflow and process associated with it. And once you have that primary structure or mental model of the process, you could go home and use that to negotiate something with your spouse.
Daniel Serfaty: I should try to use that one of those days too at my own risk. Yes, Krista you don't want to let go.
Krista Ratwani: Just to jump in and to add a little bit more to what Janet was saying. I believe Janet, you're kind of getting at levels of learning in some way, right? I think at least in the work that we do, and I believe where you can think about learning as being most effective is when you can get to that higher level. So there's what I would consider actual learning. You learn principles of negotiation, and I can tell you what they are, but then to actually go and use those to buy a car maybe that's what the course was focused on is one type of learning, but then to go home and use them with your spouse is taking that learning to a new level because then you are extrapolating those principles, applying them in an even different contacts than you originally did. So to me, in some ways that's all about building of your expertise when you can take the thing that you learned at the base level and keep applying it and taking it to a new level.
Daniel Serfaty: So perhaps the way to look at it is in term of learning professional skills, whether they are hard or soft. And again, we've come back to that definition, but it's basically different steps in the acquisition of mastery in a particular task or particular work. And so talking about that since both of you have a lot of experience working with the military, but you can certainly deviate from the military. Now with training commanders or pilots or soldiers, or even corporate managers 50 years ago, versus 10 years ago versus today, what has changed? Is it just the technology or did we understand something better about the skill acquisition, that expertise scale, or that mastery scale? Can you help our audience understand how these science of training and the resulting learning have evolved?
Krista Ratwani: So you mentioned technology Daniel. I mean, that's clearly something new and different today versus 50 years ago. And the use of technology impacts where people learn, how people learn and especially today and the times of COVID-19 learning is happening at home over your own computer. Technology has made that possible. And I think that that's key and important when you talk about training, especially higher level leaders. I also think though that the breadth of what is being taught has changed. And I think that this has to do with more of a career long focus versus I'm going to train you to be good at this particular job or train you to succeed in this situation.
All organizations, military, commercial want people who can succeed in a variety of environments doing a number of different tasks. And so giving people the skills to do that is really important. In some ways we are teaching people how to learn. So we're teaching them to be critical thinkers. We're teaching them to think about learning differently. And that I think has been embedded now in a lot of different ways, in a lot of different curriculums as opposed to being focused on that single skill.
Daniel Serfaty: So technology aside, you talked about teaching people how to learn, how to think about learning. Is that a recent discovery in the field of psychology or training science? I mean, why didn't we teach that way 50 years ago? Is that because the science has evolved to a point that we realize it's important?
Krista Ratwani: And those skills certainly aren't new and I can give you an exact citation here for an article, but they've been around. I think because of the more dynamic nature of environments where people work these days, organizations have all sorts of new demands that must be met due to competition, due to the use of technology throughout an organization not just for learning due to all sorts of reasons, I think it's letting people see that they need to focus on these skills more, to be competitive, to build the type of workforce that they want. So I don't think they're new, but I think that there's just an increased focus on them.
Janet Spruill: Right, I would agree. And I think that many years ago, training was a very institutional practice. It was important for it to be standardized and consistent, which drove it to that one size fits all training. It was also primarily in the domain of subject matter experts and what we'll refer to as the sage on the stage. Students were in a much more passive role and technology and changing paradigms have allowed students now to take much more control of the learning process, to be more self-directed. Asynchronous technologies in E-learning have supported that. But I think it is also important to talk about technology and not just learning technology, but operational technology, enterprise technologies.
The fast pace of technology is driving the need for speed to mission. So when the mission or the competition or the threat is changing regularly, so the pace of change has increased significantly. Think about cyber defenders who are watching networks and network traffic, looking for infiltration of bad actors to bring down systems and infrastructure components. We could put them through a course last month and the threat continues to change. We need to be able to provide training quickly, to be able to put it in the hands of learners, to direct their own learning and we can't take 18 months and years to push out new training.
Daniel Serfaty: That's very interesting. So in a sense, both of you coming from very different angles are looking at a very fast changing work environment, mission environment, perhaps for the military but let's say more generally work environment, whether you are a manufacturing expert on the manufacturing floor or a doctor in the operating room or a soldier in the field, everything is changing much faster. And in order to do that, we cannot train people for every single instance, but we can train them perhaps to adapt and change. So that very skill is important. And we use technologies as well as this new concept of training about learning how to learn in a sense. And I want to go back to the word that both of you said, which is soft skills. All right, Dr. Ratwani, what is a soft skill?
Krista Ratwani: Soft skills to me are those things that to use another word that you're going to ask me what it means that are more intangible in nature. And what I mean by that is they're much more subjective. They're harder to measure and especially harder to quantify. So taking critical thinking, which we've mentioned a couple of times, do I have a certain amount of critical thinking that's in comparison to something like my marksmanship score? I have a very clearly defined number about how many times I hit the target and it has a defined procedure. This is how I train it, this is how I assess it, this is how I say how good you are at marksmanship versus something like critical thinking. It's much, much harder to do any of those things that I just said. And then I also think of soft skills being much more important when it comes to that human dynamic.
So how important is marksmanship when you are trying to interact with somebody going back to my example of the military advisers. In that case, what we're really looking for is for people to be open-minded and to have empathy and to just be able to talk one-on-one with people and have a conversation. Again, all of those things are much harder to quantify, and I would argue to formally train as well. You have to be much more creative about how you're going to train someone to develop empathy than you are about how to get them to fire their weapon effectively.
Daniel Serfaty: Especially if they have to do both at the same time. So empathy is a soft skill, I understand. Open-mindedness that's another one you just mentioned. Well, you're a leadership expert, is leadership a soft skill?
Krista Ratwani: I would argue that leadership is made up of many soft skills to include all those ones that we just mentioned, to include things maybe even like communication. To me, there's a number of skills that make up leadership. Leadership is another complex construct by itself.
Daniel Serfaty: And Janet, do you agree with everything Krista say about the distinction between soft skills and art skills?
Janet Spruill: I always agree with everything Krista says. I particularly liked that she used the term the human dimension. I think that really helps to sum it up. So soft skills really are the non-technical skills is the way I think about it and how you interact with people, with your colleagues, how you solve problems, how you approach your work, how you actively listen, all of those things in addition to what Krista said.
Daniel Serfaty: So if there is Janet, you are both an expert in training, but you're also yourself a very experienced manager in corporations, what is the number one soft skill do you think is important in working in an enterprise? From what I understand from both you and Krista there is almost an infinite number of what we call soft skills that we can think about ranging from empathy to say you're honest.
Janet Spruill: That's it.
Daniel Serfaty: So which one is the one that you like?
Janet Spruill: So I like empathy. Thank you for bringing that up. I think that if you have strong empathy, you can succeed both inside your organization and outside with partners and customers because you really can see the world, see the problem, see the situation from their viewpoint, and that can help you to find a bridge, right? So you really can become a thought partner with your colleagues, with your customers.
Daniel Serfaty: And you need empathy for that. Krista, I'm going to put you on the spot too, tell me what's your favorite soft skill. And you cannot say leadership because you just taught me, it's not a soft skill, it's a complex construct. I like how those academic speak about things that are complicated. They use things like complex and constructs. But also, you cannot say empathy, Janet already took that.
Janet Spruill: Perfect.
Krista Ratwani: That's okay, I can work with those. So I actually don't know if there's one word to describe this, but to me it's finding that balance between being a good team member and being collaborative, but also taking initiative to that mix of being a leader, but being a team player. So you have to be able to work with others and you have to be able to collaborate and get along well. If you can't work in a team, that's going to be pretty dangerous in a lot of situations. But you also can't only be able to work in a team, you have to be able to take things on yourself and be willing to take things on and volunteer to lead things. So it really is that balance because too much of either of those is not necessarily going to lead to you being able to work well in an organization.
Daniel Serfaty: That's a very interesting, subtle skill that balance between individuality and team orientation. So let's talk about that for a second, because a lot of our questions and stories have to do with learning by individuals, but we all operate in teams. And sometimes not in a single team, we belong to several teams that require more or less communication and coordination and leadership, and many of the skills that competency that you mentioned. What do we do about team training? What are the big ideas in that field?
Krista Ratwani: So generally from a research perspective, team training is something that's been explored for many, many years. And a lot of it focuses on this idea of training team processes. So how do we get individuals to work together as a team? The idea that the whole is greater than the sum of the parts. So it's not just, let's add up what you bring to the team Daniel compared with Janet's skills and combined with mine and we get what we bring as a team. We really want to create that synergy. And that comes from training teams on those team process skills. So things like coordination and communication, can we make sure that we know enough about each other's roles that we can back each other up so I can jump in when you get overloaded.
And so we don't really miss a beat. We keep going. And that's where a lot of the team training research has focused. In terms of what's new, there's been a lot of emphasis on small teams, especially in the military lately. They're looking to develop smaller teams that can function a little bit more autonomously without all of the resources coming from above that were kind of this contained thing that can work on our own.
Daniel Serfaty: Like special forces, for example, or commanders.
Krista Ratwani: I think that's a great example. So kind of these smaller teams being able to work together and make decisions on their own versus those decisions coming from above and then the team executes. There's also a lot to do with what Janet talked about, getting teams to function well in these mission-critical constantly changing types of environments. We've seen a lot of that even outside of the military, things like with NASA, how do we take a bunch of people and train them so that they can go live in this isolated environment on Mars together for years without totally falling apart. So helping individuals come together as teams in those extreme environments is also something new that researchers are starting to look at and that's happening in the applied world.
Daniel Serfaty: How interesting. Janet, staying on that notion of team training, can you give our audience some examples of things that you have worked on lately about training of teams or training of team skills?
Janet Spruill: Well, the idea of team competencies and training teams skills has really gotten a lot of research attention. We're finding that programs related to customers and operational environments are much more interested today in understanding how to train and prepare teams. And that preparation certainly is from, I guess, what we would consider a pure training standpoint. So how can we train them as competent individuals and then bring them together to bring their unique skillset and their specific role so that they can operate as a high performing team. Customers are also though interested in how can we use that same data to help inform team composition and team selection. So that's a really twist that we're able then to reach back into the research area to help inform that. So a couple of examples, one in the Navy within the Center for Surface Combat Systems, they train the Aegis Combat System, a missile defense environment plus that sits on a Navy destroyer ship.
And within that, there is essentially the core information center called the Aegis Combat Information Center, where a team of personnel each with distinct roles have a number of monitors where they're watching information about what's going on in the environment. And the commander can make decisions based on that about actions that may need to be taken either offensive or defensive. So as you can imagine, it's a pretty high tempo, potentially high stress, fast moving, high stakes kind of an environment. We've been supporting them to look at not only the individual training, but also what are some of the team tasks and team competencies that need to be supported.
Daniel Serfaty: In a sense it's not enough that each one of these people in the Combat Information Center are expert like a radar expert or a breadth expert or an electronics expert, but the notion of them working in harmony with each other is important for the Navy.
Janet Spruill: That's right. And really when you think across the military and even beyond the military, most operations are carried out by teams, whether it's that elite army ground infantry teams, special forces that Krista talked to, the Combat Information Center, a team of cyber defenders who are monitoring critical infrastructure, an emergency response team or a hospital surgical team. So much work gets carried out by teams. So it's really an important area of research and application.
Daniel Serfaty: Thank you for all of these examples. I think it makes this pretty complex scientific notion of learning much more tangible because then we understand how complex it can be to work in a commander or to work in a Combat Information Center because it's not about how well you're trying to do your job, it's also how well you are trying to operate as a team. And when we talk about all these very complex environment that are complex both in terms of the expertise it needs to operate them and to succeed in them but also because of all the external circumstances of fast tempo, a lot of uncertainty, a lot of risks, I cannot help but thinking that basically in order to improve a team I need to know what to measure.
And I need to know how that thing that I measure will become better or worse or reduced as a result of the training. So I know that both of you are a strong advocate of that, but tell us again, what is the role of measuring performance in training? Then how do we use those data to improve the training, to continue the training? Why is it so important?
Janet Spruill: I think the role of measuring performance is critically important to allow us to compare a student's current level of understanding and performance against what it needs to be, because otherwise we're just making assumptions and we're not able to support that student, which is really a downfall on our part. But more importantly, we can't effectively equip that student coming out of that training to perform in their job role. So by first understanding what it is we need to measure and how those measures help inform what readiness and proficiency look like, then we can implement that and we can in real time, or at any point in the process, take kind of a track line of where that student is currently performing.
Daniel Serfaty: That's great, Janet. I cannot help but making the connection between what you just told us and differentiation that both you and Krista made at the beginning of our discussion between training and learning. In a sense, measurement is telling us if learning is taking place, isn't that right? Otherwise we would work open loop. All we can do is deliver a curriculum or teach a class to be exposed to a game, but we won't know whether or not in a sense the needle moved on the side of the learning.
Janet Spruill: That's right. And the students could come through training saying, "That was fun," but it doesn't result in any measurable improvements and ultimately then poor transfer to their workplace environment.
Krista Ratwani: So if I can just jump in and add to Janet's comment about the importance of measuring performance in relation to learning, I completely agree with everything that she said but to add a slight twist on it, I also believe that when you measure performance in relation to thinking about learning and identifying those gaps that that individual has as Janet pointed out, you can also use it to potentially identify where there are gaps in the training itself. So it may not be that somebody just didn't pick up the skill that they were supposed to through that learning experience, it may be that the training or learning experience itself was just deficient and failed to cover an important aspect of that performance domain. And so as you're measuring the performance, that gives you a chance to essentially revisit what is critical in that area. And then maybe you can go back and add something to that training experience at a later date to make sure that it's fully comprehensive.
Daniel Serfaty: So in a sense, you're saying if learning is conditioned upon the proper feedback, that feedback can be used by the learner, but also can be used by the teacher or the technology who is augmenting the teacher. So Krista, is there a science behind that? At the end of the day, we are not just proposing to measure the final score in a sense, there are other things that we need to measure. Is science helping us by understanding how to decompose those measurements so that we have a finer grained understanding of what's going on in the mind of the learner?
Krista Ratwani: I think so. And it comes down to measuring not just that final score, did they complete that task, but what happened along the way? What did that process look like? Can we pinpoint where there were failures? So the end outcome may be okay, you may have achieved whatever that goal was. You did the task, whatever the task was that you were supposed to do, but did you do it efficiently? Did you forget to do a part and you happened to stumble upon the final answer? And so being able to decompose the task as somebody is executing it in terms of performance is really important to being able to identify where those gaps are in knowledge, because if you are purely looking at it from that completion perspective, it may look okay, but you're not exactly clear how that person got to that answer.
It's a little bit like asking my eight year old to show her work when she does her math homework. Maybe she got that three digit subtraction problem correct but it was a little bit of luck. She didn't actually regroup all the numbers correctly but when she shows that she knew how to regroup it, and you can see the work there, then that really lets us know that she understands the process behind it.
Daniel Serfaty: In the '80s and '90s, there was a lot of development around the technology called intelligent tutoring systems and intelligent tutoring systems were basically a way to acknowledge that each student is different and each student learn at his or her own pace. And therefore, if we could use technology by being intelligent, therefore adapt itself to the students, we may solve the problem of very large classrooms or cookie cutter training that works for everyone the same way, et cetera. So today we call these collection of technology basically the second or third generation intelligent tutoring system for adaptive training. Can you try to view perhaps share with our audience your own definition of adaptive training here? What is it and why do we believe it's a game-changing technology, Janet?
Janet Spruill: So I like to think of it in fairly simple terms. To me, adaptive training is a data-driven approach to learning, which means data is the keyword there. But the beauty of adaptive learning that smart use of those data allows us to do some pretty powerful things like continuously modify the training content and modify it based on both the behavior and the needs of each learner. So the behavior could be that I have stalled and I'm just simply stuck. And the needs could be that I come in as a more experienced learner. So perhaps I am a military service member that's had a deployment or maybe I am a very proficient healthcare professional, but now I need to learn the very specific skills of intubating a patient, which might be a new skill.
So it's able to modify based on both the behavior and the needs and that could look really differently for different students. But generally when we talk about adapting, we can adapt the sequence of the content, we can dial up or down the difficulty based on how that student enters the training or how we're measuring their ability to proceed through it. But we can also adapt the timing and the type of feedback to help make that student more successful.
Daniel Serfaty: Thank you, Janet. Keep those in mind because I want to come back to almost each one of those dimensions in a second, Krista from the scientist researcher view of the world, what is adaptive training?
Krista Ratwani: The way that I like to sum up adaptive training is that we are personalizing that learning experience to give the learner what they need when they need it and in the manner that best reflects how they're going to effectively absorb it. So the what goes back to the content that Janet talked about, the when refers to the timing, which is critical, you want to give somebody content before the skill that they learned decays but you don't want to give it to them too early when it's too challenging for them. And then the how really comes down to the method and that could have to do with just pure logistics of it. I'm only in a place where I can have my phone with me. So I need to be able to get it in a mobile environment or it could have to do with the learning strategy. I don't absorb information well by reading, but I can certainly get it when I see a good visual of it. So being able to be very personalized in that approach so that the learner really can just get what they need from the content.
Daniel Serfaty: It's amazing as both of you are giving your own take on adaptive training, personalized training in a sense. I'm thinking of two things. I'm thinking in a sense, we are reinventing what those great teachers of the past already knew. Your mother teaching eighth grade probably knew that each student learns English differently, has different abilities, different preferences, different levels of attention depending on the topic and was going from table to table from desk to desk, to in a sense adapt her teaching to each one of those students. It's difficult to do if you have 30 students in a class. So what you're saying is that we're trying to replicate this extreme personalization and deep expertise that great teachers have into understanding sometime intuitively, sometime by experience what each students need in order to progress optimally. The other analogy that I'm thinking about is that many of the systems we use these days are the fact of personalized, because as Janet says, it depends on the data.
We cannot have high levels of personalization without data. Can we? Like Netflix knows what is the next movie I'm going to like most, how do they know that? Not just because they take a snapshot of the movie I'm currently watching, they have my whole history. They've locked in the sense of model of the users. So we seem into our industries that are being redirected towards an equal one, the teaching is not for the class. So Netflix experience is not for the population, it's for a single individual. So what are then the necessary ingredient you started to list them, but in terms of data, if data is the key ingredient. So of course there are algorithms that can process that data and then we redirect, as you said, to a particular path of learning, what is necessary in terms of data to close that loop? Janet, I know you're a big fan of data as a major enabler here.
Janet Spruill: It most certainly is. And I think to hone in on a part of your question, what are the ingredients for adapting and personalizing training to succeed? I hone in on that word succeed. What does success and part of the success or an enabling part of the success in adaptive learning is all about the ability to gain individual student insights just like that teacher does who walks from desk to desk and then act on those. So if we don't have individual student insights, we can't act on anything. And so that requires the availability and access to data as one of the most critical ingredients for that.
Krista Ratwani: I would add to that, the more that you know about that individual learner, the better off you're going to be. And that's things about what that learner knows, of course, what they don't know, kind of as I alluded to earlier learning preferences. But then I would say the other critical piece of data that you need is about that learning experience or learning content itself. What is that content trying to teach? What's the learning objective? How hard is it? How does it actually go about teaching it? Coming back to the modality and certain modalities being more or less relevant depending upon the situation are more effective. And so it's really knowing about the person you're trying to get to learn or train as well as what is that training material and thinking about the training environment itself.
Daniel Serfaty: What's your both thing, which is I think essential is it's not just at that moment of the teacher's intervention or the intelligence systems intervention, it's not just the instant data that matter what happened in the previous few minutes, but it's a whole history in fact of that student. So we can achieve levels of precision that are incredible if we can not only collect those data, but also know what to do with it in a sense process it in ways that we know what to do with it, is that the real promise of adaptive learning, or does it go beyond that Janet?
Janet Spruill: The promise of adaptive learning covers a lot of ground. I think that one that's really important, especially in these times to talk about is the issue of learning equity and driving learning equity at scale. And what I mean by that is there's a fair amount that's published about the potential of adaptive learning technologies to drive equity at scale on higher education. But I think it goes more broadly than that. I come back to think about the personnel that we interact with, the young Navy recruit and how adaptive learning can help these young sailors who come from all backgrounds and all achievement levels. So whether it's in a college or a military technical school, these underserved students can feel perhaps disproportionately unprepared for many aspects of their learning experiences coming into them. I think adaptive learning can really serve almost as a great equalizer to help them succeed and build a lot of confidence as they go through their learning career.
Daniel Serfaty: That is fascinating, Janet. I haven't thought about it because what you're saying is precisely by hyper focusing precisely on that individual, we might be able to achieve a societal goal of equity.
Janet Spruill: I think so. I mean, we all want to be understood and in a learning context where we have an instructor, a teacher, we want that teacher to get us and to meet us where we are. Using technology, we can do that in a very empowering and self-directed way.
Krista Ratwani: I guess I'll back that up maybe with some data. So I think that's a really great insight, Janet. And in some work that we've done at Aptima with our own adaptive learning technology, we have some data recently from an internal research project where we basically looked at who does adaptive learning help. And when you think about the traditional classroom approach to training, it's kind of designed to help that student in the middle, that average student, right? Because they have to teach to the class, not to the individual. And so you have to set it at a level that's appropriate for that average student. And you assume that most students are going to be able to grasp it. And what our data show us is that in the experiment that we did, the adaptive learning conditions were beneficial for those students on either end of the spectrum.
So those students who came in with a lot of knowledge and those students who came in with a little knowledge. And Janet, I think in your comments, you're speaking to maybe those students who didn't quite get what they needed, they are kind of on the lower end of that knowledge or skill level. And so the data that we have illustrate that that's exactly the group that we can hope to help, and maybe the middle students are going to be fine with that traditional approach and don't necessarily need that extra help.
Daniel Serfaty: That is a fascinating way to look at how advanced algorithm and artificial intelligence that goes into those adaptive training systems in order to benefit from this extraordinary large reservoir of human data is actually able to do things that have always been very difficult in schools enterprises is to take care basically of both sides of the normal Gaussian curve. That is an extraordinary insight. So Krista and Janet, thank you so much for spending the last hour with us. You really taught me and my audience quite a bit about learning, really opening our horizon about that very unique human endeavor, which is the acquisition of new knowledge and new skills and perhaps now is generalized to societies and machines and everything else.
Thank you for listening. This is Daniel Serfaty. Please join me again next week for the MINDWORKS Podcast and tweet us @mindworkspodcast or email us at mindworkspodcast@gmail.com. MINDWORKS is a production of Aptima Incorporated. My executive producer is Ms. Debra McNeely and my audio editor is Mr. Connor Simmons. To learn more or to find links mentioned during this episode, please visit Aptima.com\mindworks. Thank you.