Why ABA Training Fails, and How to Fix It (with ABA Technologies)

Most ABA training is not really training. It is compliance theater. Staff click through the modules, pass the quizzes, get signed off as competent, and six weeks later they are standing in front of a real client, in a real moment, with no idea what to do. We did not skip the training. We did plenty of it. It just never taught anyone how to do the job.

My guests today are two of the most respected voices in the field on exactly this problem. Dr. Alison Betz is VP of Business Development at ABA Technologies and the current executive director of the OBM Network. Adam Hockman is their senior vice president of Learning Partnerships and Growth, an instructional designer who will happily tell you that ABA Technologies is not a plug-and-play training company. They are presenting a session at the Summit called From Training to Results, and this is the episode to listen to with your notes app open.

Here is what we get into:

• What plug-and-play training really is, and why a 40-hour checkbox bolted onto an already bloated onboarding produces content coverage, not competence.
• Why instructional design is a discipline of its own, and why being brilliant at teaching kids does not automatically make you good at teaching adults.
• The sequencing trap. Why the skills a new technician needs on day one, like pairing and building rapport, tend to show up last in their training.
• Why you do not have to choose between exam pass rates and real readiness. A well-built program can deliver both, often in less time.
• How to actually run the build-versus-buy decision, starting with a number most owners cannot name: what training truly costs you, including the billable hours you lose while people are still in it.
• The measures worth watching. Time to competency, 30, 60, and 90 day turnover, first-session integrity, and the satisfaction surveys behavior analysts tend to wave off.
• One thing you can do before August. Follow a single new hire through onboarding like a researcher, and map exactly where the confusion and the handoffs are.

A few resources from ABA Technologies if you want to go deeper:

Company page: https://www.abatechnologies.com

RBT Cost Calculator, to put a real number on what training is costing you: https://www.abatechnologies.com/corporate/cost-calculator

The ABA Tech Clinical Leaders Community: https://www.abatechnologies.com/corporate/the-aba-tech-clinical-leaders-community

Alison and Adam will be on stage at ABA C.A.R.E.S. for their session, From Training to Results, and they will have a booth right when you walk in, to the left. Adam's advice for getting the most from the Summit: come with two or three real problems you are trying to solve, not just one, and be willing to look underneath them, because training is often the hidden factor behind a turnover or compliance problem. 

Register here: https://behaviorlive.com/conferences/abacares2026/registration

We will see you in Boston August 4th through 7th, or live streaming from the other side of the screen.

Transcript

Copy Transcript Copied

SPEAKER_02 0:00

Hi, I'm Holly Beth. Welcome to Inside ABA Cares. Your behind the scenes space where we set the expectations and lay the foundations for what you can expect this August and who you're going to meet. And so training in ABA is strained. We know that. Not because we don't do enough of it. We certainly do. We do plenty. But it's more because what we call training is actually just more of compliance theater, check boxes, if you will. Staff click through the modules, they pass the quizzes, BCBAs sign off on competencies. And six weeks later, when they're on the floor, they're still unprepared for real moments in real life. And so my guests today are two of the most respected voices in the field on this exact problem. Dr. Allison Betts is VP of Business Development at ABA Technologies, Executive Director of OBM Network, Adam Hockman is their senior vice president of learning partnerships and growth, instructional designer, performance architect, and the person who proudly states that ABA Technology is not a plug-and-play training company, which we're going to dive into. Their session at the summit is called From Training to Results. And if you're an owner or clinical leader who has spent a lot of money and time and not seen it translate to confidence and competence, then this episode is where you want to have your notes open on your phone. But before we get into it, I do want to get to know you guys a little bit better. Alison, I'd love to start with you.

SPEAKER_01 1:31

Yeah. Thank you for having us. We'll make one slight uh adjustment so people a little bit about myself. I have been a board certified behavior analyst since 2005. Born and raised in Michigan and got my PhD at Utah State University and has kind of been everywhere in between. Currently uh working at ABA Technologies more now on the organizational behavior space and helping employees and helping organizations with improving employee performance in general. We take the full 360 approach from the individual all the way to organizational and cultural experiences. So it's been really fun working and taking what we're doing at ABA Tech into the clinical space where my roots have been planted and where I've gotten all my training.

SPEAKER_02 2:26

Awesome. Well, I can't wait to dive in and explore that more. I know it's a full onion that we want to peel back. But before we do, Adam, I'd love to hear from you as well.

SPEAKER_00 2:35

Sure. Well, excited to be here and excited to be uh presenting at the ABA Care Summit. I've spent most of my career as an instructional designer and a program implementer for various organizations, but primarily to serve children and families, like schools, clinical ABA. And so at ABA Tech, I spend most of my time working with clinical organizations and helping them look at their onboarding and employee experiences through the lens of training, clinical systems, measurement and evaluation, and building programs and systems that improve that.

SPEAKER_02 3:08

Great. Before we go further, a lot of people when they hear instructional designer, I feel like picture more what the graphic designer would be. So implementing the beautification of slides and things, but that's not what it is. So if you can tell us a little bit more about what instructional design is before we get into some of the questions.

SPEAKER_00 3:31

Sometimes you find it inside of the broader area of learning and development in a lot of companies that might have a HR department and then they've got an LD function, or LD might be on its own. But instructional designers or people who have experience with adult learning theory can think about building learning experiences that are going to help people do their jobs better, at least in the corporate and organizational sense. And so they've got a good pulse on models for how to design instruction, how to evaluate the outcomes of learning programs, how to see whether or not your training or learning program is having an effect on certain kinds of outcomes that you want to achieve, whether that's business results, whether it's clinical quality, those kinds of things. And a lot of people fall into instructional design. They work in other areas. They might be teachers or they might be someone worked in a role and then they moved up to a trainer position. And then eventually they started developing training. And along the way, they joined some of the professional associations like Association for Talent Development or the Learning Guild, where they learn about best practices in adult learning. And that's true in ABA. A lot of organizations don't necessarily have instructional designers. They have training directors. And most of those training directors are people who started their careers as BCBAs and then moved into a role of maybe trainer and then eventually training director. And so they're figuring out instructional design as they go along. And some of them don't even know that they're doing instructional design. Call it training because that's what they learned in their supervision course.

SPEAKER_02 5:04

I'd like to hear from your expert vantage point. What is something that you would recommend when somebody is looking at a plug and play?

SPEAKER_00 5:12

Aaron Powell First of all, define at least what I think of when it when we talk about a plug and play training solution. And that's something that has been built probably pretty generically by a training vendor like what we are. Or in compliance, this is very common. So an ABA organization may work in multiple states, and every state has a different requirement for sexual harassment or HIPAA or something of that nature. And so they go out and look for people who've already developed those training programs, like an online course. And so it's a course that's ready to go, and so they plug it right into their training program and they hit go on it. And so a lot of people in ABA do that with the BACB's 40-hour training requirement for RBTs. They work with a digital data collection platform, for example, that has an embedded RBT training, and then they use that to support meeting that 40-hour requirement. And there's nothing wrong with that necessarily, but oftentimes it is treated as a compliance requirement rather than the kind of learning that's going to help that individual do their job. And because it's treated that way, it gets inserted into an already pretty bloated training program. So if you think about the RBT experience, when someone's onboarding as a new hire to become a technician and then maybe eventually becoming an RBT, they're going through HR training. They may be going through instructor-led training where a facilitator is teaching them how to use a data collection system, how to look at a programmed description, how to identify the controlling prompt for starting a program, those kinds of things, things that are specific to that organization. And then what they'll do is they will add in either as homework on that day or throughout that week or after they finish the their brief onboarding experience. They'll add the 40-hour training. And it will be this plug-in play. We'll just lift this 40-hour training. It meets all the requirements. So that's good. It's an easy checkbox, but we'll insert it and folks will be on their way. And so it ends up becoming this content coverage rather than an opportunity to practice, get feedback, apply skills that they're actually going to use on the job with clients. And you'll often hear this. This is the way that organizations talk about it. It's the way that supervisors and trainees talk about it. They say, I have to do training and then I got to do the RBT modules. They see it as a completely separate thing. And because it's been so separate, a lot of people, the way they've developed it, is like how they've developed state compliance requirements. When you go through a sexual harassment training, it's almost always a set of slides that someone's reading the information. It's hitting all the main points they need to hit, but there's not relevant practice in there. It's not sequenced in a way that's going to build upon that employee's entry knowledge and experience related to the topic. And so in RBT, that happens quite a bit. You'll watch people go through very passive learning where they'll have very few opportunities. They'll get to the end of 30 minutes, 20 minutes, whatever it might be. They'll answer a couple of questions and then they go on the floor and they work with clients and they're doing the things that they were supposed to be learning in the online program. They just can't do any of it because they didn't practice any of it. So it looks like all that training was really irrelevant to what the job is. And then we're upset that that things aren't going well. We have low procedural fidelity with clients, all those kinds of things that come up that supervisors have to do a lot of retraining. They have to deliver a lot of feedback, do a lot more observation than they expected. And then they have trainees that never become fully independent. And as a result, that requires way more support from them. And at the same time, turnovers happening. And so new people are coming in. And so you're managing new people at the same time as people who've not met competence in six months who are still there. So it's a big challenge. And sometimes you don't always see it because if you're sitting from the training director role or a or compliance role, again, plug and play is very easy to do. When your whole 40 hours is an online course that people complete a module exam after each one, it's so easy to show somebody that they've followed all of that and that you are fully compliant. But it doesn't often lead to the outcomes that we want in practice.

SPEAKER_02 9:29

Yeah, it's interesting to me because it's designing the turnover problem from the beginning. So you have a lot of individuals that we hire, especially at the BT level, who may have never worked in this space before, certainly don't know ABA most often, or even working with individuals with autism, and they're trying to push through these 40 hours so that they can get working with the client. So oftentimes they can even fast forward and watch on 2x or whatever, but they're not really processing it. And to your point, there's no behavioral skills training, there's no feedback to that. And so what does a healthy alternative look like? Allison, you've worked across different fields, clinical ABA, fleet, healthcare, corporate. What's one thing that every industry is getting wrong about training?

SPEAKER_01 10:22

That's a good question. One of the major things that I see across all industries, including those in human service and ABA therapy, is that people tend to think that training is a single event. It's a singular thing that happens. And so I complete the module or a workshop, whether it's live, virtual, on demand, and it's done. I get a certificate, I've done check that box, like we talked about. Really, the way that we think about it at ABA Technologies, and what I have come to learn through working with our wonderful instructional designers, is you have to look at training as a journey rather than a singular event in space. And it should be ongoing, it should be continuous. It's not going to be just this one piece in time of I watch this module done. I know what I need to do. It has to be designed with that journey in mind. And so thinking more about it as this ongoing progression. And yes, we want to get initially for onboarding, for example, with technicians, we want to get them up front to the most independent place that we can for them to be successful in that very first session by themselves and be able to complete their core responsibilities, but it doesn't stop there. It has to continue. There has to be this continuous learning journey. Even 40 hours, the completing a 40 hours, the best that you can do it. And you follow best practice and do design that optimally, there's still going to be more to come, right? And people think, oh yeah, they did that training. They should know how to do it now. And they forget, especially if you're doing it in an online on-demand way, that there's going to likely be this knowledge skill gap. So yes, they might be able to regurgitate what they have, but they may not be able to demonstrate it in actual practice across the board. That's what I tend to see. With clinical or ABA specifically, that is a piece of it. And we see we just need to get it done. This we're having this one workshop type thing, and all our problems will be solved. In addition to that, I think the biggest thing that I see in ABA organizations specifically is that they don't understand the importance of knowing instructional design and the understanding how adult learning is very different than working and teaching their clients. And when I I first got here and started working with Adam prior to coming to ABA Tech, a lot of what my job was incorporated training and doing the RBT and tech training and ongoing BCBA training. And he probably heard it more times than he wants to. But it blew my mind starting to work with him of like, oh my goodness, what I was doing this all wrong before. Sure, it could have been a little effective, but was it as efficient and optimal as it should have been? Probably not. And so I think clinically with behavior analysts, especially, we pride ourselves in understanding learning, right? That's what we do. We know how learning occurs. But I may venture to say that when we're going from your initial training and competence, working with young children with autism, for example, saying now that you're competent in teaching adult learners in an organizational setting may be practicing out of our scope of competency, right? We don't know the best practice, the specific research. It'd be similar to saying, I know how to work with a nonverbal three-year-old with autism. Therefore, I can go work with an adult with a traumatic brain injury. You have to understand the population and the context in those idiosyncratic variables that may uh change some of the procedural. Yes, the core science is the same, but there are other aspects that you have to understand and become knowledgeable about. So that's what I tend to see, I think, because we almost are, as a field, overconfident in our understanding of learning and behavior and how to change behavior, but don't take into account those very unique variables across the different populations.

SPEAKER_00 14:29

There are a lot of people who believe because they've been trained in the science of learning, they know learning theory, they can apply it across the board. And I'll also say that at one time that did work in a lot of ABA settings, like in the 90s and the early 2000s when ABA was really small, and it was very possible for well-trained clinicians to be on the job, coaching people in the moment with clients. Clinicians were spending a lot more time as the direct implementer rather than supervising a bunch of implementers who've had, you know, at most maybe 40 hours, 60 hours of training. And so as ABA has scaled, it still tried to rely on that very high-intense training model, uh, for example, using behavioral skills training or focusing primarily on the use of feedback. And when you are primarily focusing on those high effortful training practices that are more individualized, and you start to scale, then you start creating a lot of gaps and you can't do that at scale. The kind of training that's still being used today did work when you had a clinician who had a lot of experience, was trained in a high-quality university program, had experience in a university clinic that was well supervised. Then they go out in the field, they'd work with clients, and they may have some therapists that work with them or families, caregivers directly implemented interventions. And at that time, they were able to use behavioral skills training. They were able to be there on the job to really train and coach people. And that works when you're really small. But as you start to scale, take on more clients, you're multi-location, multi-state, that becomes a real big risk for an organization. And the more that you lean on really intensive training models like that, the more work you have to do to maintain quality and the greater risk that it poses as you continue to scale. So at one time that worked, but that model does not scale well. And that's why people in instructional design, learning, and development that have been doing it in corporations are able to do it quite a bit better because they know how to deliver it and deploy learning at scale.

SPEAKER_02 16:36

You both point out um things that are important to pay attention to. I want to double-click a little bit on what you just said, Adam, that made me start to think about back in the day in the 90s where you're mentioning we were able to be more hands-on, obviously things have changed. One of the things that changed was the introduction of needing to be certified, right, as an RBT. And so now we added an exam component. And what we've seen, and I'm curious if you've seen this as well, is that people are training to pass the exam, not training for competencies. And so I'm wondering how that shift has impacted uh the approach to training, if at all.

SPEAKER_00 17:19

Well, I can give one very real example of how I've seen it when I look at other training providers, is in the way that they sequence content. Many training providers I've seen, and I haven't looked more recently, but with the third edition test content outline and the update to the curriculum guide, they will literally take the curriculum guide from the BACB and use that as the framework to deliver training. They'll group content by those different categories. If we were writing a textbook and we were trying to organize it by chapters, that might make sense because we could put everything that's about the three-term contingency, we could put that into one chapter, and we could put everything that's about preference assessments in another chapter, and we could group them that way. That would be great if someone were taking a three-credit graduate course on ABA because they are in the class for 16 weeks and they're not necessarily expected to go out in the clinic and deliver intervention. But when you're coming in and you're going to be working as a direct implementer or therapist, you are expected to start pretty soon as you get hired to work with that child. And the first thing that you don't need to know is the history of ABA, or you don't need to be able to identify every consequence. You need to learn how to effectively build rapport with a child. You need to be able to carefully watch their indicating response or be able to capture and contrive MANS. And so a lot of the things that people need to do very early when they actually are working with clients are not the things that come early in their training experience. And so I do think that that's one of the changes with the RBT requirement is that people are trying to meet those marks. And what they're doing is they're creating really poor sequencing and poor scaffolding, and then everything breaks down from there because those people then enter the floor and they're working with clients and they don't have any of the skills that are going to help them immediately build rapport and start contacting reinforcers in doing the work itself.

SPEAKER_01 19:17

There's a balance, right? Because we do, as working with the leadership and the company owners or the CEOs, they need those people to pass the exam to be able to bill. And we understand that's important, but there is a balance. And this is one of the things that I learned quite a bit about when starting to work with Adam and our instructional design team is you can do it in a way where it you can get them to pass the exam too, and have the more on-the-job real-world experience and learning experience that's going to help them on day one. And so it's not one or the other. And so I think when people are like, yeah, but I need them to be able to pass the exam. Well, yes, you can. And we have demonstrated working with clients that we can build this and scaffold and drip the content and reorder it and maintain high pass rates in often shorter amount of time for training time. So I don't want people to come across and be like, yeah, but if we start with the pairing and doing some of that, then it's going to push and delay their ability to sit for the exam. That's not how an effective program would be built.

SPEAKER_02 20:22

Yeah. I think that's a fair concern that you're calling out. Which I think drives some of the decision to go for a build versus um buy. Is that when they're thinking about it, if they can just buy it and get it started, that's fine. They'll worry about the competency down the line when we're thinking about how to help owners when they're looking at this and potentially underestimating the value in a good training program or trying to figure out even how to balance the cost of training, especially in a high turnover industry, they are scrutinizing the cost of training to no end. And so when you're talking to those owners, how do you help them when they are trying to decide which way to go? Yeah.

SPEAKER_01 21:08

This is a big question that comes up a lot and is one of the things that we have to focus quite a bit on with leadership. And I think there are several questions that need to be answered internally to make that decision. You touched on several of the important points, cost being one of them. So the first thing that I actually do, and surprisingly, not a lot of leaders know the answer to this, is say, how much does it currently cost you to train? And you can take the the hourly wages and the salaries and the background checks and the certification costs and all of those things and get a rough idea. But you also have to take in some of the lost opportunity costs, right? So if it's taking you longer to train, how much are you losing in terms of billable hours? So if you can get them trained even 10 hours sooner, like a one working business week sooner, what are you losing? If it takes that much longer, right? What are you losing in terms of quality of care and not just staff turnover, but potential client turnover as well? So I think when we help the leaders of the organization understand what all goes into the cost of training and it's not just those hourly wages. So that would be the number one question is do you know how much it costs now? If not, let's go figure that out. And we actually have resources that can help people figure out how much it costs per RBT, a rough estimate if they don't know. Then from there, I'd say, okay, we know the cost. Now let's talk about the resources. So what does it take to build a and redo or design an effective quality program, training program? And do you have those resources? So do you have a team of people that you can spare that maybe that's going to take reducing their billable hour requirements to be subject matter experts to help build? If if you're lucky enough to have an entire training team or department that they aren't required for other responsibilities, what are your other initiatives? And how many people do you have to be able to get this done in the timeline that you want? So can you afford to do this across two, three, five years, or does it need to be done sooner and what amount of time is allocated? And really laying that out to see how much non-monetary resources you need and whether you have those. And then adding up what else is going to be delayed if you take on this project internally. And then the expertise. So do you have the expertise? And obviously, most ABA companies, if not all, will have the subject matter expertise in what they're teaching. Is that great? What prompting they want to teach and what they're going to teach for error correction and naturalistic or preference assessments. They have that content. But do you have somebody who knows how to design and build the trainings in an effective manner? And that's where we see going back to what we talked about, that's where we see the biggest deficit. And yes, you might have somebody who happens to know some instructional design. Are they able to take on this entire project or do they need additional supports? Right. So those are the three main issues. And then after we talk about that, we want them to really think about the risk of poorly designed training. So yes, you can just plug and play. I can buy something or build something. I want it done quick. All we need is these 40 hours of modules. Give me PowerPoints. However, doing that, understanding what the outcome is going to be, is you're likely going to get poor engagement. You're going to get high turnover with those technicians, burnout with your BCBAs because they have to provide so much feedback and supervision and they can't even pay attention to the actual client management portion. And then, of course, the quality, right? I mean, that's the biggest part. We're all here to make sure that our clients make good gains in their in quality programming. You can have the best behavior analyst in the world programming, but if you don't have a well-trained and independent technician, it doesn't matter, right? That fidelity is going to be lost. So we try and walk through those steps and asking. And it may be that the answers to those questions does lend to they build it on their own versus outsourcing or a hybrid approach.

SPEAKER_02 25:25

When I've been brought out as a retention expert to look at companies and what they're doing, I've been shocked by how many have what you call subject matter experts who do understand what they're trying to teach, but quite often it lives in a binder that they say they can refer back to and that the training is done because it lives in this binder that somebody can look at. For the individuals who are running the organizations, because we do have a lot who are trying to do really good work and they lean heavily on their clinical team to tell them what is needed for this training. What are some green flags that they could listen to to identify if they do have that talent in-house or if they need to be looking outward?

SPEAKER_00 26:10

If you've got clinicians that you think might be people who are going to be training designers, I think that's important to make a distinction that there are people who are going to maybe design training and they're going to people be people who deliver training. And they're two very different functions. And many places, if you had a learning and development department with multiple staff, you'd have instructional designers, you'd have facilitators, you'd have people that develop instructor-led training and make it package it for facilitators to develop. And so if you were looking internally to see if you have that expertise, you're wanting people that when they bring new hires in, they're the ones that can kind of give them the hidden curriculum, if you will, of how the job really works. And they can describe it in a way that's going to make sense to them. If you teach someone how to run a particular program and they're confused by it, can they actually reword it in another way? Can they give different examples? Can they say, let me try it? And now you do it, and I'm going to make sure that you get some additional practice with another client, those kinds of things. So things that they're very attentive to the features of what needs to happen in order for a new person to start building that awareness around what they're going to be learning and what they're eventually going to be doing. And I I think it's also important to look at their technical precision, their verbal behavior about the science and practice of behavior analysis, because they're going to be writing clinical standards that are going to get implemented by the organization. You want to make sure that those are robust and based off of best practices that people are in contact with the most recent literature and so forth. And then that they have good skills themselves. So many people that are now being trained in organizations and then move from like an RBT role to a BCABA role to a BCBA role, they move through that progression so quickly that they themselves don't have a lot of client direct experience. And it shows when they are then coaching a new person on how to play with a child, it looks like they've never interacted with a child. You look at them, try to build rapport, they can't generate creative ideas of how to use a play set. You know, those are some of the things that I would be looking for because those are the people that are not only going to be able to design and deliver training well, they're the people that are going to get new hires excited about the work. You want to look for those people that get really excited when a client passes off on a goal or you see that you've set up a shaping progression and they've moved through many steps of that shaping progression. Those are the people that you want to have in training roles. And it doesn't even have to be formal roles. They don't have to have the official title of training director or trainer, but someone who can be a champion for training and onboarding and building that experience for people. And when we talk about whether people should buy or they should build it on their own, Allison brought up a lot of points that you need to consider. I wish more organizations would buy and work with training providers that have experience in instructional design and then reallocate their staffing to be champions in the change effort. Because anytime you're implementing a high-quality training program, people are going to have to change. So many times that we've developed training for companies, they say, we're going to have to go back and train all of our behavior analysts on how to do this. You know, if we're going to upskill the RBTs, we're going to have to make sure the analysts are at the right level. So it's no small feat to make changes to training. And that's why people don't do it this way, because it's hard and it requires people to change. And it doesn't feel very good and it doesn't look great on paper when you're in the change and you have to go through change management and all of that. And so it can get very messy, but uh I promise the the rewards are there. It's worth doing.

SPEAKER_02 29:52

You're leading me into my next question. I think that logically we can all understand the reasons for this, but uh, do you have a case study that you can share you've experienced firsthand? Yeah.

SPEAKER_01 30:04

One of the ones that we've recently been promoting and presenting is in combination with Little Leaves and Dr. Ivy Chung with what we uh did to help them rebuild their RBT training. And they came with us, they have about, give a brief background, like 14 to 18 centers across what, three to four states right now. And they were growing. They had about 300 RBTs that uh were coming in per year. Um and it was taking them a really long time to to get through training. And when we went through and kind of did our deep assessment and and understand what was happening, what we identified was when they work doing what we typically see of that check the box, let's do the 40 hours and then they come to us and now let's really train them. Some of them even said they do the 40 hours and then then training begins. And it but what Adam was referring to is they look at it as two separate things. And it was taking, I think, what Adam, it was like up to like on average, I think it was like 120 something hours to get through. But they were getting good treatment fidelity. Their initial session check was high. It was like 95%, but it was taking them so long. And so their rationale to this was really um, it's yes, it's taking long, but we are able to uh get the the results that we want because they're it's high quality. And Ivy came to us and was like, can't we have both? And so we worked with them to redesign and do some of the things that Adam was talking about, looking at the order and the schedule, how much is done instructor-led and live versus sprinkling in those on-demand modules, what content should be in module-based versus live, when the on-the-job shadowing is happening and when things like that are occurring. So we redesigned that entire experience and worked with them to make sure it was the constraints of what they could do as well. Because that's really important of understanding we could build an amazing program, but if they can't roll it out and sustain it, then it's not gonna matter. So, with that, what we ended up finding is we were able to decrease their training time. I think it was from like 125 hours to it was like 90 something, correct, Adam, if if I remember correctly. And with that, we were able to save about $440,000 a year in training costs for them just by decreasing. And that's in wages alone. We didn't even take into consideration some of those lost opportunity costs. So legit just how many people they were training and the time that they were taking. With the reduction, I think it's important to know, even though we reduced the amount of time of training, we still were able to maintain those high fidelity. So we were able to keep it at that 90 to 95 for their first session check. The other thing that was amazing, and I think is probably one of the best outcomes that we've seen, not only the cost with in reduction in time with maintaining quality, we were able to reduce their 60-day turnover. So we did a pilot program and looking at the turnover of the people who went through the new training versus the original, I believe is 23% for the baseline group to 8%. Um, so what that shows in the 60-day turnover really does is a good representation of the uh the engagement and the satisfaction with training and the support within that first little bit. So we're fairly confident that the the training itself, that change, is what uh improved their engagement, their overall satisfaction and feeling more supported. So they were ready when they were independent. Adam, is there anything you want to add to that?

SPEAKER_00 33:45

The other thing that I would say is that part of that engagement, and this has happened with other organizations we've worked with, they get to work so closely with our team of instructional designers, behavior analysts, educational technologists, they get to see training from a whole new perspective. And it really has cascading effects. There are organizations we've worked with in 2020 who are still applying a lot of the main design and development and delivery strategies that we taught them then. And I remember uh going out and doing a pretty large train the trainer with some RBT trainers several years ago, and I introduced them to the field of learning and development, introduced them to this job that is training and let them know that there are other people in other organizations not in ABA that are professional trainers. And that was huge for them to find a community of practice that was beyond just working as a as an RB because their their roles have changed. And there are a number of people in ABA that maybe don't always know that, especially if they've moved from RBT to trainer, that training is a profession and best practices related to that profession and their ways to engage with it, uh, find community, improve your practice and uh their conferences, all those kinds of things. So it really opens doors for people in terms of a career progression.

SPEAKER_01 35:03

Adam brought up a good point of what we did with Little Leaves and what we do with the with the companies we we work with. We look at the entire training program as a system as well. And so it's not just the content of the RBT, but what we're looking at and what Adam did that I think allowed little leaves to have so much success, worked very closely with training those trainers and making sure that they we could again, we can give you the most wonderful content, but delivery is a huge part, understanding how to get that engagement. And so a big success, I think, of Little Leaves was that we really prepared the trainers, both the ones who were doing instructor-led training, kind of that more didactic in the classroom, and their on-the-job training, the shadowing turned into very active learning session and working with the people who were responsible for the trainees during that time and making sure that they were prepared and knew what the expectation was, how to give the feedback, what content to cover, how to do that. So I do think that that is a huge contributing factor to this success.

SPEAKER_02 36:11

You're walking the walk. If you're going to train and have champions, as you've mentioned, those champions have to understand the full scope of training. And I firsthand experienced that back in, I think it was 2017. I was sent to the Association for Talent Development because I was a trainer at that point. And it did open my eyes so much so that it eventually helped me name my my dog, who is Addy, after the instructional design model. So it's very close to my heart. And I think it also helps me when I'm thinking about the conference. And as I'm building that out, I have this in the back of my mind. And so I like so much that you guys are mentioning so many threads that are great. When you guys are talking about champions, do you help them identify trainer?

SPEAKER_00 37:00

Well, in the example that Allison gave with Little Leaves, that's a very human-dependent training program because there are facilitators, trainers, there are people that are going out and doing observation and feedback. And so you really need those trainers to be on board with it. It starts at the top. Ivy, and this is not just true of Ivy, but we did a big project a number of years ago. And uh Hannah Roo is the chief clinical officer there. And uh Hannah and Ivy, when we worked with them, they were in every single train the trainer session. They were sitting at the table with technicians. When we taught them how to do choral responding, Hannah got up and got coached by me to do choral responding. Ivy did the same thing. So those people being in the room and being vulnerable enough to say, this is new to me. I know a lot about training. I know a lot about developing clinical systems, and I know a lot about doing clinical work, but this is a new area to me. And having trainers watch their clinical leaders engage in that is really big. And that will tell you whether or not this thing's gonna sustain. So the first thing is if they're not willing to do that, be in the role of the learner and the trainer, that's a good indicator of how much they're gonna champion it. And one of the exercises that I like to do with people when we start working on a training project is having them sit in an existing training session. You go through and now you sit in this training on prompting and you tell me what you practiced. How many times did you? I want you to tally, how many times did you respond? Did you get to did you ask questions? Did they call on you for questions? Did they only call on one person? How was the pacing of the session? Then we'll reteach that content in the format that we're gonna deliver it, where it's very practice scenario-based, it's very active. And they say, wow, look at the difference between those two. Uh, you know, I really learned something. I had to be very active with it. And we do a lot of that with trainers that we're gonna work with. They first are learners. Um, and in the example of Little Leafs, we first ran a train the trainer where I trained all of them in the content they were going to train on. And they had to be learners. And so they got to experience all of that, reflect on it. They will end up giving you the messages, the change messages that you can use when you start rolling it out. They'll give you the testimonials, they'll give you the words to say, the difference that it made for them, those kinds of things. So we really look for those people that see and notice a difference in the quality of the program and the difference in the way that it's designed. If they can detect those features, they're more likely to be advocates for it as it starts to go through change. And that's really important because for a lot of people, the change is not gonna be easy. And so you want to look for those people and you want to see them dispersed across the whole organization. We'll do uh sessions with sessions with leadership so that they can start seeing what we're trying to do is affecting one piece of this broader performance context. And what we don't want them to do is think that, oh, we're gonna come in and flip the switch on training. We're not gonna fix any of the other problems that are in place, but we're gonna have this huge expectation of a change. And so we want them to see that it's a context. Inside the context, we're working on this one pillar, but these other things are gonna have to be at play at some point, maybe not all at once. And if they can really lock into that, then you also have advocates on your end that can really support the change.

SPEAKER_02 40:26

Yeah. It's really robust and you leave really nothing to chance. And uh, you covered a lot of some of the misunderstandings in the training industry that plagues ABA organizations right now. Is there anything else, though, that you'd like to add?

SPEAKER_01 40:41

Going off of what Adam was just saying, I mean, look, making a change like this and revising your onboarding or upgrading, whatever you want to call it, designing is a big feat. And you have to look at it through the change management lens. And I think that's where if you get some leaders when they come to us and they're just like, even if they're going to engage with us and like just can't you just like put it in, like just build it and and drop it. And we actually will say, no, we we can't because we know it will not be successful. And so I think the the biggest thing on the more of the systems organizational side is that this is an entire ecosystem in itself. And it you can't just do this one thing of drop or change out the slides and expect any kind of full change. And you're gonna have people who might for push back a little bit and like the old way because it's easier and it's what they're familiar with. And it's about the messaging, it's about the entire process of change and rollout and implementation. And those are the things that also need to be considered that that we do help uh leaders and sponsors and stakeholders walk through each of those steps. And because even if you have leadership on board, if the messaging, if the implementation, if that rollout is not done correctly, you can get people who are very much against it. And we all know a couple naysayers can really dirty the pot pretty quickly, right? So other things to consider in thinking very strategically about not just design, but the entire implementation rollout as well.

SPEAKER_02 42:18

It's a heavy responsibility to bear. I remember when I was a trainer, I had with great power comes great responsibility, the Spider-Man quote. It is a lot of responsibility when we are talking about training and making sure that it's implemented right. But when a training fails, or when an individual is not able to put it into practice, who's responsible? How do you figure out where the breakout breakdown is?

SPEAKER_01 42:45

I have, I will get on my soapbox. We are behavior analysts. And so if something fails, we have to look at the environment and the context in which that failed, right? And so I personally think, and maybe a lot of leaders might not be happy with this, but it is the on the organizational level and the leadership level where we have to, they have to take accountability and look at it as where did we go wrong? How are we arranging this environment and the contingencies in a way that is not supporting effective learning or practice, right? And again, it might be you have great training and there's other areas in which something is failing. So they go through the training and they're still not competent or they're not demonstrating this skill. It might not be your training, it might be the other contingencies involved. And we see this a lot in fleet industry and safety, right? People will know to drive the speed limit, and yet drivers are still not doing so. And so I personally think that we have to start at the organizational level and start with looking at the context and the environment in which the behavior is occurring and understanding and assessing why it is or is not happening, why that desired performance is not occurring. And at that Where that OBM lens comes in of understanding you have to look at the individual performer, but you have to look at the processes and you have to look at the systems in which those processes and performer are interacting and understand how the organization is supporting the behavior that is actually occurring. So my position would be the responsibility lies in the leadership and organization. Now it may be that they can delegate to understand a little bit more. And from there, if it is the performer level, there's one set of actions. If it is process and system level, it's another. But it's their responsibility because they are the ones responsible for the environment that the performer is in.

SPEAKER_02 44:42

Excellent answer. Adam, did you want to add anything to that?

SPEAKER_00 44:45

I completely agree. I put in my notes the learner is always right. And we usually think about that when we're working with a client, but we completely release that philosophical stance when we're working with adults. And for, you know, I often tell trainers, you know, your learner used to be the client in front of you, and your new learner is the trainee. And you need to see them as the one that's always right in the way that their behavior is going to tell us a lot about the environment. And to the best of your ability, you're going to try to arrange that for them to be successful. But to Allison's point, so much of the time we intervene so low on the ladder, like we're intervening at the therapist implementer level. That's usually where people come to us because they have problems at the implementer level. And they don't really, they haven't really looked carefully at what they're doing in leadership to create that context.

SPEAKER_02 45:37

Which makes me think of the owner operator or a small organization that feels like, but I'm very hands-on, right? And sometimes it's hard to self-identify why something's not working. Is there a headcount or number where you shouldn't be looking at this? Or how do you help small clinic that that maybe is too close to see it and is listening right now?

SPEAKER_00 46:00

Yeah, we've worked with organizations of all sizes and from a from a headcount perspective in terms of being able to scale training. I don't know that there's an exact number where the system can break down, but you can see issues when people begin to scale. Oftentimes the clinical system setting up standards for how people are going to deliver clinical care, how they're going to evaluate the outcomes of that, those are not well established. You may have the owner who's sitting in every single rounds, you know, grand rounds or group supervision meeting where they're hearing all the cases, they're hearing what's going on, they're hearing about issues with families and so forth. And then as you start to add clients or an or a supervisor leaves and another one comes in and they're not performing in the same way or don't have the same history as the one that just left, you find yourself in a spot where you have to stop everything. And I know that from several people that I've worked with who have been in that situation where they had a supervisor that was there for a very long time, they left and they basically had to put everything on hold, their growth for the clinic, adding programming, adding an after-school programming, all those kinds of things, expanding services. So there's a lot that can happen if the owner operator doesn't have their eyes on the right thing. And sometimes when we've met with organizations of that size, I have often thought, why are you concerned about X or Y? You should be really concerned about Z. And sometimes that's really hard because if you're the person that's taking out the trash and you're also wiping down all the tables and you're running the clinic and you're supervising everybody and you're developing the programs, you kind of just get in the rut of doing everything. And then all these other pieces become these sideline initiatives that you never really get to. And so you don't ever get to see the outcomes you want with them. Allison and I met a couple of years ago at this point with a very impressive owner-operator of a very small organization. And she came in, she knew exactly these are my three targets for this year. And she was able to describe all the actions she was taking, aligned to each of those. And she was relentless about looking at the things that were getting in her way to keep her focus right there. And that's what I think a lot of folks in that situation should be trying to do.

SPEAKER_02 48:19

People do feel like they're running out of time and they do recognize that a lot of things live in their head, but they don't have the time to get it out and to get it down, or even potentially they are writing down what their targets are, but they can't get it across the finish line. And so we see a lot of people looking at AI in that respect, being like, I'm just gonna type it all into Claude. Claude will figure it out, they'll write it, and they don't have time to review what Claude says. They just put it in another system and says, and they say, okay, cool, make some slides off of this, and we're gonna run with it. And I'm gonna hand it to somebody else to figure out. What's your reaction to the AI training content that's flooding the market right now?

SPEAKER_00 48:59

Well, there's a lot of it out there. Not as much of it has maybe entered the ABA space, though I have seen quite a number of misuses, not so much in training, but in dissemination of information because of exactly what you said. Uh people are asking some AI platform to generate a lot of content. They're not giving it a lot of information to go off of. Their prompts aren't very good, and then they don't have time to review it. And sometimes they're completely out of scope as the subject matter expert, so to speak, in that role. They're designing content for which they don't have subject matter expertise. And so they're looking at the output as something that is this looks pretty good. It looks pretty accurate. And I think that's actually a part that is very scary, is that a lot of the stuff that's getting out there in terms of content, both in clinical organizations, but also in graduate training, is sort of right. And a lot of people have not fully developed competence in a lot of areas of behavior analysis to discriminate between sort of right and actually right. Now we're just taking that sort of right stuff and training people on that. And so it doesn't, you know, you've got people that don't have the subject matter expertise on the content, but they also aren't reviewing it from the lens of learning design, implementation, continuous improvement, and those matter just as much. Um and so you're very likely to get yourself in a situation where you generate a lot of stuff and you're like, oh, this is great. We've got so many slides, but it's not focused on the aim of changing performance. It's not focused on the aim of teaching people specific skills. So definitely legitimate uses to it in instructional design and training. It's very helpful for brainstorming scenarios, generating practice activities, generating some copy, checking text, those kinds of things. But you need really skillful people looking at that material so that it isn't inaccurate, disconnected, shallow, those kinds of things.

SPEAKER_02 50:57

When people are using AI, they feel like they can say, act as a subject matter expert in instructional design. And since they don't know a lot about it and they just again press enter, is there a risk in doing that?

SPEAKER_00 51:08

Then it's just as like hiring someone who says they're an expert in something and you take them completely at their word and you're not critically thinking about the output and looking at it. So there's a big risk in that people will, when they're generating content, they're not looking at it carefully, and they've they've given a prompt like you you suggested, is that they'll look at it through the lens of, well, it must be right because I don't know anything about instructional design. So that must be the way to do it. And so even if you are using those tools to try to generate content and you're using those kinds of prompts, still keep your eye on what the goal or the outcome is going to be. If you are wanting to generate content to teach people how to effectively implement prompts, look at whether or not they can do that and look at whether they can do it under a variety of conditions, under different programs with different learners sitting at a table on the floor in the hallway, more complex, less complex, across time, all those kinds of variables that we would look for for any performance we're trying to train. And if as a result that content's not working, that training's not working that was generated that way, that's a good measure of whether or not you should continue to do that in the future.

SPEAKER_02 52:20

Which leads me to you, Allison, is looking at training through an OBM lens. What is the measurement that an organization can be tracking to know whether or not the training's working? Yeah.

SPEAKER_01 52:32

There's some of the in terms of obviously training completion and any kind of pass rate. So if we're looking at RBT, first-time exam pass rates, if that's the main goal. Also first session checklists or independence, right? So regardless of certification, that's one thing, but it doesn't talk about practice and in on the job competency. So finding a way where after training, your first session independent on the job, simple competency type checklist or a treatment integrity checklist of are they doing those core critical skills successfully, um, will give good insight into how well that initial core training did. Uh, some of the others that I would highly recommend that will help get into the cost piece of it is the time to competency. So this isn't just time to completing your 40 hours or your how long is it taking them to be truly competent in being independent in one of those sessions where I could have my first billable schedule to a full billable schedule. And then we look at turnover, so 60, 30, 60, 90-day turnover will be huge indicators of how your training is going, as well as those other costs, right? So the cost of training in terms of your salaries and your wages for your trainers and trainees, but then those opportunity costs. So looking at gained or lost billable time, the number of clients that you're able to see with training. So how many billable hours in utilization can you get with the trainees when they're done? Now you get into some of the harder ones, but I think they're still doable in terms of engagements. So one of the things Adam had mentioned when talking about having the trainer be the learner and sitting in and measuring things like how often are you responding? How often are the trainers asking questions and the people are actually practicing? You can take that data and correlate it with your outcomes, right? For some of that engagement and satisfaction. We, as behavior analysts, underestimate the importance of satisfaction questionnaires and surveys and understanding how they liked it. We do interviews with technicians when we're working with clients during our assessment. And you hear so many times people saying, I just, I'm so bored. I don't get anything. I'm sitting in front of a computer spring for two weeks. I haven't even talked to my supervisor yet. I haven't even been on site yet. So some of those things you can formalize and see what they are thinking and getting some insight into their satisfaction and level of engagement. I'm really working on now post-training stuff of how well prepared did you feel? So after training, you're you're done with the training, you are checked off and ready to go. But do you have the confidence that you will go in and do this well? So some of those satisfaction measures are really important, and you can take those as broader stroke of let's keep an eye on them. It's not our primary measure necessarily, but if we start seeing some dips, maybe this is when we need to start open the curtains and looking deeper into what's going on. So it might be kind of just a flag measure, right? But it's super important. So those are the ones I highly recommend.

SPEAKER_02 55:51

For the people who are listening, they already have their tickets and they're coming to Boston or they're gonna see you on the other side of the screen on the live stream. And we are hoping that they'll listen to this to be able to prep for your talk, knowing what they're gonna get out of it and have their expectation laid. Is there anything else that you want to share with the listeners that they can already start to think about or prep before they come to your talk in Boston?

SPEAKER_00 56:17

Sometimes people will come to an industry conference like this and they'll have one goal. They're very focused on one issue and they're gonna get every single talk that's on that issue, and that's gonna be the only thing they're gonna focus on in networking and so forth. And I would encourage people to, as they think about attending the summit, think about expanding that to two or three organizational problems they're trying to solve and not just, oh, I'm I need to solve a turnover problem, I need to solve a compliance problem, a supervision problem, but thinking more underneath of what is actually contributing to that problem. Because sometimes people will not think, oh gosh, training is a factor that's impacting turnover. Maybe I should think about that or I should talk with other people about what they're looking for. So a number of those things are really important to start coming to the summit with, and then you can get a lot of out of it. I'd also say that if you don't have a good pulse on your current onboarding experience and to what Allison was, knowing people's satisfaction about it, oh, between now and August, I would encourage people, if you've got a new cohort coming in, even if it's only one person, is to be sort of like a qualitative ethnography researcher. Immerse yourself in their training experience. Ask them questions along the way, track how long it takes them to complete things, put together an onboarding journey map to look at where handoffs are. Is there confusion? Are there questions? You will learn a lot when you dip down into someone's experience, and that'll tell you a lot about what you're looking for. And then finally, Boston's a great place. I lived in the Boston area for a while. So Back Bay, Seaport, Clover Food Lab, Atlantic Fish Company, Lolita's. So lots of great restaurants and things to see. Take the Boston Harbor sightseeing tour and all that.

SPEAKER_02 58:07

I love it. Well, Allison, Adam, thank you so much. We know that most of the training, it feels like something we do to staff more than an experience. And what you did was just shine a really hot spotlight on the complexity of all the richness that can come out of it when training is done right. And so two totally different approaches. And I'm excited for people to come and experience yours. And I'm sure they already have a notebook full of things written down, like I did when I was listening and talking to you, next to all my hearts of things that I loved that you said. Anyone listening, please come find ABAC Technologies team. They're getting in Boston August 4th through 7th. Alice and Adam will be on stage for their uh talk training for results. They'll have a booth right when you walk into the lap. So walk in and find them, tap them on the shoulder and ask them the questions and just start the conversation with curiosity. So we will see you in Boston or from the other side of the live stream. And thank you again so much for joining. Thank you for having us. Thank you.