Ep. 63: 5 Data Collection Mistakes BCBAs and Special Educators Should Avoid

Are you drowning in data but still missing progress? In this episode, we’re talking about the five most common data collection mistakes we’ve seen (and made!)—and how to fix them. Whether you're a BCBA, teacher, or therapist, these tips will help you streamline your systems, support your staff, and actually use the data you're collecting.

🧠 Key Takeaways:
📊 Too much data = too little connection — Prioritize quality over quantity.
🧍‍♀️ Inconsistent systems can lead to confusing or unusable data—align your team early!
🧾 Vague definitions = meaningless data. Use clear, shared operational definitions.
🧩 Wrong tools = no follow-through — Choose formats your team can actually use.
⏰ Data with no review is wasted — Schedule regular check-ins and use what you gather.

🧰 Resources & Links
      🎯 Caitlin's IEP data tracker forms • Audra's FREE data forms via the weekly newsletter 
      🧩 Related Episodes: Misfit Minute for the best clipboards • Episode #16 Probe vs Trial data • Episode #13 Keeping Materials Organized

🤝 Join Us
      💬 Let us know your favorite data collection tip in the Misfit Behaviors Facebook Group
      🌟 Subscribe: Don’t miss upcoming episodes on practical strategies and must-have resources for behavior support!
      📢 Share this episode with a teacher, BCBA, or administrator looking for fresh approaches to special education and behavior analysis!
      💬 Let us know what YOU wanna talk about in our Facebook group  

Join the Facebook group for collaboration and freebies: The Misfit Behaviorists

😍 More, you say? We’re here for you!

• Apple podcast | The Misfit Behaviorists
• Instagram | @‌themisfitbehaviorists
• YouTube | @‌themisfitbehaviorists

👋 Find us!

• Audra | ABA in School
• Caitlin | Beltran’s Behavior Basics
• Sami | B.A.S.S.

🖱️ Rate, Review, Like & Subscribe so you don’t miss an episode! Showing this love helps us get out to more educators out there!

Transcript

Ep. 63: 5 Data Collection Mistakes BCBAs and Special Educators Should Avoid

[00:00:00] Caitlin: If you're working on a behavior plan and you've been running it for a while, it's running really well in terms of fidelity, but the behavior is not decreasing. Is this the right behavior plan or do we ne need to tweak it? 

[00:00:12] Intro: Welcome to the Misfit Behaviorist Podcast. Join your hosts, Audra Jensen and Caitlin Beltran here to bring you evidence-based strategies with a student-centered focus. Let's get started.

[00:00:25] Caitlin: Hi, everybody. Welcome back to the Misfit Behaviorist Podcast. Today we are talking about five common data collection mistakes and how to avoid them. So before we start, just a reminder to head over to our Facebook group. We are putting a lot of free content in there and then have a lot of good discussion threads going on, so definitely check that out. So I'm gonna dive right in. So I definitely have made all of these mistakes before. So starting with number one, one of the most frequent mistake I see in classrooms is collecting too much data. So Audra, have you or anyone you worked with's been guilty of this before?

[00:01:00] Audra: Oh no, not at all. Ever. No. We always collect the exact, oh my gosh. This is probably the one we've done the most. We love the data and we love the graphs and we love seeing it. So we get over exuberant about what we're taking and what we're expecting other people to do.

[00:01:18] Caitlin: Yes, and we have been trained, like that's why we're here, right? Like we're here for the data. Like we're data driven. We're data based. Those become our mantras. And so for sure, I think all of us at one time or another have been guilty of this. I know the second part of what you said is totally true for me, of expecting others to be able to take too much data. Like when I first moved from my teacher role to my BCBA role, I was like, you can probably get all this done today. And so I feel I've grown over the years in this and reprioritize just quality trials over quantity and good teaching and getting to know your students and building relationships, not at the expense of data, but just maybe shifting the priority between those two.

[00:01:58] For example, you are not going to track data on every single goal for every single student every single day. That's never going to happen, and it shouldn't happen. If you are doing that, something probably is getting sacrificed along the way. So quality time with students, some natural learning opportunities. We do not wanna just be data machines where it's like 1, 2, 3, 4 plus done Overall.

[00:02:19] Audra: You just miss so many opportunities. You're like, your face is on the clipboard so much you're trying to take the data. You're missing the opportunities of doing natural teaching as well. It's just you're constantly doing this or on your iPad, then you're not interacting with the student, which is so important.

[00:02:35] Caitlin: And the last thing we wanna do is overwhelm ourselves or our new staff members and lead to that burnout or like you said, miss the forest through the trees, like everything's happening, but all we're doing is charting it and not really responding to it, or, I've had that happen sometimes with behavior data, where if I realize the staff are so busy trying to tally every instance, okay, this can't be the right format for this, because yes, the data is important for sure, especially with behaviors, but we also need to be in the moment and responding and promoting better choices. So I think the tip for this would just be to focus on those high priority goals and then set yourself up with some good systems that work for you. Keep it manageable. If you find yourself collecting too much data or at the expense of other things, take a step back and think about, do I need to collect every data point or can I move to some kind of probe? Can I do data twice a week? Can I do data in the morning? Like anything like that. We've experimented a lot the past two years with those things in my classroom, and I think we've found some good systems that work for us as far as moving, shifting overall to more probe than trial by trial data. Then also shifting to for certain learners, we just do data days once or twice a week, and we are in school a lot of weeks, so we still get really good data.

[00:03:52] Audra: I would add as a tip here too, to be okay with putting down the data, if you see something happening, to give yourself grace, like, oh, today is a data day, I have to get this data done, but you know what? Something's happening. Be okay with just putting it down saying, you know what, I wasn't able to do that, and that's okay.

[00:04:07] Caitlin: And also just trusting your staff. If you are in a position where you're the teacher or BCBA and you're a little more removed, I think my worry in the beginning was if I say it's only Tuesday, Thursday's data, the staff are gonna forget. And the exact opposite has happened every single time with me, where my staff are coming to me saying, I know it's Tuesday, but there's an assembly. Do you mind if I take it just the afternoon? And then also we'll do tomorrow morning. And I'm like, please, like I did not think that would be the problem I would have. So just trusting your staff to do those, make those judgment calls too.

[00:04:38] But I think moving into mistake number two would be that inconsistent data collection. So this is making me think of classrooms where I've had six to seven paras. Like I've had a lot of students and I've started to realize, oh, you're tracking the data this way, but you're tracking the data this way. And most often I would see this when I started to look at the data and be like, that is so weird. Like how is this learner getting this a hundred percent some days but never on other days. And so when we start to look back, we might realize, oh, you thought it meant with a visual, but you thought it meant independent. That makes sense because they're definitely getting it right with you and they're always getting it wrong with you. So not to point fingers or just say anyone's doing anything the wrong way, it's just a misunderstanding. So the tip for that would be to set those really clear routines, expectations, like try to write down notes as you can, but also role playing with the group when the students are not present. And then if you are the teacher or BCBA in charge, really observing not just the students but the staff taking data and making sure you're getting that kind of inter- observer agreement between each staff member as much as you can.

[00:05:46] Audra: This is a really good place to make sure you have that operational definition really defined. Whatever behavior you have them taking skill or behavior that it's really clear so that you can hand your little statement of what you're tracking to anybody and they're looking at the same thing. So that just helps. Just make sure that from the beginning that you set that up so that all the staff collecting data are collecting it on the same thing and then that inter- observer agreement is so important. And even if you just have the two paras have one watch the other one. Then back and forth and just say, Hey, did you see the same thing? Take it at the same time, it doesn't have to be all the time, but just periodically touch base and make sure everybody is looking at and tracking the same thing.

[00:06:24] Caitlin: Yeah, so we segued right into number three, which is that vague or unclear data definitions. I feel like this happens in my classrooms more with behavior data, but certainly happens with skill data as well. For example, I'm saying, oh, we're tracking off-task behavior. But if I didn't define enough examples or use that perfect operational definition, you might have two or to three staff members tracking this differently. It's just we don't want the data that we are taking to be subjective or really worthless if we're all tracking it differently. So just making sure we're using those really clear operational definitions, especially for behaviors, but also for skills. For IEP goals, I think it comes into play more when , are you expecting independence? What types of prompts you're using, what types of visuals or materials you're using? I've seen learners be able to count perfectly with rubber bands, but not with paperclips or vice versa. So maybe that's an example of how that role play could be really important. If I'm saying like, guys, we're using these materials, I want them to generalize everything. Don't just teach in this way and making sure that all of your paras are then following that model.

[00:07:29] Audra: And how frustrating is it if you've clicked a whole lot of data and you go to look at it and you have to throw it all out? Yeah. Oh, we lost it all. 'cause that doesn't make any sense now.

[00:07:38] Caitlin: So definitely save yourself that feeling and try and get ahead of it when you can.

[00:07:43] Number four mistake, I would say, is just using the wrong tools for what you're trying to track. If you're using like a really complicated spreadsheet, you've developed, oh, we're gonna track behavior, frequency goes up here, and then duration goes up the bottom. Mm-hmm. And then we can also track the trigger. It all sounds great in theory, but we've all had those data sheets where you're like, this is a masterpiece data sheet, but in the real world, no one's gonna be able to use this unless you're just sitting in a clinic observing, which is rarely what we're doing. We don't wanna use overly complicated systems that are gonna increase errors or discourage that para or staff member to use it. We wanna use the most simplified version that captures the most of what we wanna see. So if you have a data sheet that has more than two things, or you're tracking two different ways that might work for you, but it has to fit with who's tracking it and the behavior or the skill you're tracking.

[00:08:36] Audra: That's a really good point because when you're talking about data collection, is to make sure you're the people who are collecting data, that it works for them. So it may work beautifully for you and you understand it and you're super digital, but maybe this particular teacher is older and hasn't done digital tools, so you need to think about what works for them. Bringing your data collection that you've come up with to the person who's collecting the data and making sure that they're comfortable with it, they understand it, they're gonna do it. Because if you set up a system for them, that is the perfect system in the world, if they're not able to do it or don't want to do it, it's gonna be useless to you. Just making sure that the recipient of collecting the data is cool with whatever you're setting up.

[00:09:14] Caitlin: A hundred percent. And again, I've had that go both ways, like I've had sometimes where it's too complicated or too much for a staff to use. Then recently a teacher was telling me I had made like a very basic probe sheet and she was saying like, but I really wanna note like they do it wrong, it's with this picture or that, and so there wasn't enough room for her to write the notes on it. Okay. I was like, all right, maybe we can to the middle on this, we'll change it. And then I've also had to tell staff, I love that you wanna record everything that happened. It's not useful, but that's also, it's not useful. And that's such a horrible failing to tell staff because I feel like, of course it's useful, it's beneficial, but again, like

[00:09:49] Audra: data overwhelm

[00:09:50] Caitlin: it's data overwhelm and it ties into what we talked about last week, I'm picturing all those notes you wrote down, we could have had a conversation with the learner or we could have had more face time or more relationship building. So just making sure we have that balance and data and face time with our students.

[00:10:04] And then finally, mistake number five. Definitely I think the most important and the one we are most guilty of, not following through, not reviewing our data regularly or at all.

[00:10:16] Audra: Pivoting and pivoting with that

[00:10:18] Caitlin: making those decisions. Absolutely. We're all really great at taking the data. I think we've all made mistakes one through four on some level, but from the whole, we're able to find that data sheet that works for that staff. But at the end of the day, at the end of the week, at the end of the quarter, what are we doing with that data? And if the answer is nothing. Why are we even taking it? And sometimes that's not a bad thing. Sometimes that leads us to a conversation. We've been tracking this for a while. We're at a standstill. Maybe the answer is not do more with that. Maybe the answer is put that data on hold and focus on something else. But whatever we are tracking systematically and spending so much time perfecting that data sheet and that system, and we've role played, we really wanna make sure that we're looking at that. It doesn't have to be every day. It doesn't even have to be every single week. On some consistent basis, even for a few minutes at a time, so that we are able to use it and make those data driven decisions. If you're working on a behavior plan and you've been running it for a while, it's running really well in terms of fidelity, but the behavior is not decreasing. Is this the right behavior plan or do we ne need to tweak it? If you're teaching counting, and this learner is consistently not meeting criteria with counting one through five, they're not ready to move on, and we need to actually dig deeper and figure out why. Is it the materials, is it the learning style? Is it the prompting? So even just a few minutes at a time, glancing over your data sheets, whether you pick one student a day to look through. Or pick your Friday afternoon prep to glance through the binders. That to me, I think was important just to set a system that worked for me. My supervisor taught me early on, you have to schedule that just like you schedule a meeting. If you think to yourself, oh, I'll look at the data when I have time this week, it might never happen. But if I put that on my calendar as Friday afternoons at one o'clock, which is probably the worst time to look at data, let's be honest. It should maybe earlier in the week, earlier in the day with fresh eyes. Tuesday at 10 30 is my data time and I just glance at it for half an hour, that can really set myself in a good position for the rest of the week.

[00:12:23] Audra: I think that's a really good tip to move into, you're running your IEPs as well, is looking at your goals when you're writing a new IEP that you're not just taking a goal that hasn't been mastered and just moving it on the next year or taking a goal and just dropping it off to really look into that data and really tweaking those IEP goals that are legally responsible that we are meeting and not just flippantly take and leave this or that. Mm-hmm. Really be thoughtful about how to change an IEP. So it's actually useful for that student.

[00:12:51] Caitlin: And not that we wanna think about, like, we wanna think about the data for doing the best for the child and their progress, but the reality is too, just like you're saying, we are accountable to that IEP, to that child, to that family. So you never wanna have that conversation at the end of a progress report marking period, whatever it may be. Oh, don't worry. We practice it every day. Don't worry. We took all the data and then stop there like we wanna say, and we looked at the data and it was useful and we changed this or that, depending on progress.

[00:13:19] Audra: And then we have funding sources outside of that. In the clinical world, if you're not making progress or if you can't show data and why you're making changes or something, you may lose funding for that student and that student needs that therapy. So that's really important as well.

[00:13:34] Caitlin: And sometimes that data shows us maybe we just pick the wrong goal. Exactly. Like I've had students who aren't progressing with skills. It doesn't mean we have to pull our hair out every week and try a new prompt. Maybe I have to go back to the drawing board and pick a new skill that they're more likely to progress with.

[00:13:47] Audra: And I like to ask, is this goal going to benefit this student in the long run? Yeah, if they're stuck on identifying coins and they've done this for years, how important is it for them to say that's a quarter? If they're gonna eventually just be able to use a debit card, is we really think about how is this going to be important for this particular student?

[00:14:06] Caitlin: Moving on to a more functional goal. So that's it for this week. Those are the five data collection mistakes that I have seen most often. Again, I've been guilty of all five. I'm sure.

[00:14:16] Audra: Totally agree.

[00:14:17] Caitlin: And as I have put systems in place that have worked for me, hopefully some of those can work for you.

[00:14:23] Don't forget to check out our Facebook group. Head over there. And definitely if there's a topic you wanna hear more about, feel free to shout it out, and that's where we get most of these topics. So we'll see you over there and hear from you soon next week.

[00:14:35] Intro: Thanks for listening to the Misfit Behaviorists, and be sure to tune in next week for more tips and tricks. Don't forget to subscribe so you don't miss an episode.