Ep. 4 Beyond the Messy Desk: Assessing Executive Function in Adolescents

Think a messy backpack or desk is just carelessness? We don’t think so. This episode cracks open the complex world of executive function in school, showing therapists how the Weekly Calendar Planning Activity assessment can unlock insights into how adolescents may be struggling to think, plan, organize, and navigate their world. Plus - Lacy and Michele answer a listener's question, exposing a sensory-based myth found on Instagram.

We share our own thoughts in the Research Review and encourage you to read the article too.

Tamm, L., Hamik, E. M., Zoromski, A. K., & Duncan, A. (2024). Use of the Weekly Calendar Planning Activity to assess executive function in adolescents with autism spectrum disorder. American Journal of Occupational Therapy, 78, 7801205040. https://doi.org/10.5014/ajot.2024.050295 

Keywords: Occupational therapy; OT; evidence based practice; OT ideas; Peds OT; pediatric occupational therapy; executive function; assessment; autism

Stay informed, stay curious, and stay playful!

Email us at IdeasAtPlayPodcast@gmail.com

Find us on Instagram @ideas.at.play

Transcript

Michele: Welcome to Ideas At Play where we discuss pediatric research and help you apply the ideas to your daily practice. Each week we'll review evidence based ideas to make you a better therapist. I'm Dr. Michele Alaniz, Director of Pediatrics at a rehab hospital in Southern California. And with me is my good friend and former coworker Lacy.

Lacy: That's me. I'm Dr. Lacy Wright, an occupational therapy professor in Kansas City. We're peds OTs love research and making it fun. helped thousands of therapists just like you to become more informed. playful and more effective in their sessions.

Michele: Today we'll be discussing executive function and the Weekly Calendar Planning Activity. Let's get started.

Lacy: If you love what you hear in today's episode, please sign up for our free weekly newsletter, where we [00:01:00] provide a quick summary of the research and links to any products or places we discuss in the episode. Email us at ideasatplaypodcast@gmail.com.

Michele: And up first we have our Nailed It or Failed It segment where we discuss what worked and what didn't in our practice this week because sometimes when you're trying to get that just right challenge, well, you're just going to fail it. Lacy, you're up first with our failed it.

Lacy: Oh, Michele, I ruffled some feathers with the school district admin this week and relearn the importance of good communication. Because this is still fresh in my mind, and honestly, I'm trying to get back into their good graces, I'm going to share a similar story that happened a few years ago. 

Michele: Okay. Wow. So this is so fresh. You don't even want to talk about it, but it's a recurring issue for you. So let's see if we can problem solve this together.[00:02:00] 

Lacy: Yeah, clearly, I have had to relearn my lesson. So. I had this grand idea to make a CrossFit activity circuit like a recreational space attached to a high school classroom that I was serving. And

Michele: Love i!.

Lacy: I know it was a great activity. I was brainstorming it for weeks. I was thinking about how many IEP goals this would help support. I had my supply list. I was hyping up all the students and the staff. Then, I submitted the supply list to the special ed director and she's a couple supervisory levels above me and it caught her completely off guard. This set off a massive email chain with all these names that were cc'd on it that I had never seen before. Tons of meetings

Michele: No, not the CC.

Lacy: I know, the CCs of Doom, um, and budget [00:03:00] discussions. I had incorrectly assumed that the teacher, who I was planning with, had already cleared the idea, and clearly, it was not. 

Michele: So then when it ended up happening,

Lacy: Fortunately, I ended up getting the supplies, and it all worked out. But it could have gone a lot smoother. And what I learned, and wish that I had remembered last week, when I got into the same trouble, to ask “Who else should we bring into this conversation when planning ideas?” And, yeah, that would have been better.

Michele: That's good. That's good. That's a good lesson.

Lacy: Yeah, there's always someone else that needs to be looped in. I need to do that.

Michele: Well, you know, for me, I like value efficiency and making decisions quick and like moving fast and breaking things. And so I've gotten myself into that trouble so many times.

Lacy: Yes, okay, Michele. That means you are up with a Nailed It this week. Lift us up and [00:04:00] tell us about it.

Michele: I love my Nailed It because it happens to somewhat correspond with the whole CrossFit theme. I don't know. I guess we're like channeling our inter Yovana. Yovana is a good friend of ours who used to be really into CrossFit and would do it with the kids. I have a client that I'm working on self regulation with, but he is, minimally verbal, still learning how to communicate with his AAC. So it's not like I can, you know, do zones of regulation or the alert program or something like that with him. I'm really trying to learn his triggers and see what gets him worked up and then how we can. Keep an eye out for that so that we can intervene and help him before he kind of goes over the edge.And so when he came in this week, I put my Apple watch on him and I started a workout and I monitored his heart rate while we were doing different activities just to see how his internal autonomic nervous system [00:05:00] reaction was to the different things that we were doing. And it wa so awesome, because I could see like, oh, he's getting pretty worked up just based on the heart rate going up because we weren't doing a lot of active things. We were working at the table. So as his heart rate was going up, I was able to preemptively help to calm him down and bring him back and keep him engaged that way. So I just loved it. It was so much fun. And then also, he already knows the strategy of trying to take a deep breath. So when he starts getting worked up, he tries to do that, but it ends up being like a hyperventilating breath and

Lacy: Yeah.

Michele: It gives you a haptic cue is like where you can feel it touching or tapping you, but there's no sound to it where it can walk you through a deep breath and really emphasizing that longer exhale for him. So it was just a win win with the Apple watch.

Lacy: Wow, that's a [00:06:00] great activity. I never thought of using an Apple Watch that way.

Michele: The only downside is Apple watches are expensive and if I tried to go to the special education Departmental aid and said you need to buy Apple watches for everyone. I don't think that'd go over very well. So

Lacy: I think it would trigger a lot of meetings and the CCs of doom.

Michele: Exactly. I really like the way our nailed it and failed it went together, even though we didn't plan it ahead of time. You know, great minds think alike. And that is it for our nailed it and failed it segment this week. 

Michele: And now on to our research review, where we break down the latest in pediatric research and explore how it applies to our sessions. This week Lacy has an article on executive function and the Weekly Calendar Planning Activity. Lacy, tell us all about it.

Lacy: This article caught my attention because executive function is a pretty hot topic, there's a lot of information about it. However, it's really challenging to explain and to quantify.

Michele: [00:07:00] Lacy, before we go on, let's just talk about what executive function is because I know I had always heard that term. It wasn't really, really a hundred clear on what it meant. Um, but now I've done some education on it and I understand that when we're talking about executive function, we're really talking about three core elements, working memory, inhibition, and cognitive flexibility. And then there are a bunch of other skills that play into that, but those are like the foundational executive function skills that help us with planning and prioritizing and doing goal directed tasks.

Lacy: With middle schoolers and high school students, it's important for academic and school planning activities - planning large projects, especially projects that take more than one day, Students juggling time with sports, friends, homework, keeping a backpack or locker organized and just keeping track of due dates for assignments or tests.

Michele: I think you can always tell how good a kid's executive function skills are just by looking in their [00:08:00] desk or looking in their backpack or looking in their, uh, school notebook.

Lacy: Yes, it is so revealing. 

Michele: So this study was looking at executive functions, which we all need. So I'm excited to hear about the Weekly Calendar Planning Activity. I'm not familiar with that.

Lacy: The article was written by researchers from Cincinnati Children's Hospital Medical Center and was published in AJOT last year. It's called Use of the Weekly Calendar Planning Activity to Assess Executive Function in Adolescents with Autism Spectrum Disorder.

So we'll jump into the weekly calendar planning activity and then we'll talk a little bit more about how they used it.  This assessment can be really valuable in assessing your kids that you might find like right on the bubble on whether or not you want to recommend OT services or not. It's great for kids that can do a lot of like basic routine tasks, but struggle more with complex or non routine [00:09:00] activities. One more thing about this assessment is it's pretty inexpensive.For AOTA members it's only like 115 and that gives you everything you need. The testing forms are digital and you just print out as much as you need. You don't have to keep buying protocol sheets.

Michele: That's nice. Executive function can be hard to get a test for that's an actual direct measure. I know in the research, the outcome measure they most commonly use is a proxy report where a parent or a teacher is rating the child on how they're doing.

That's called the BRIEF. And I feel like if you can direct measure it, it's just it's more accurate. It's more useful. Of course, the questionnaires give you data, but it's so nice to have something you can directly give to a child.

Lacy: Yeah, it's interesting. Actually, in this article, they compared their direct measure from the weekly calendar planning activity to the BRIEF. And there actually wasn't much of a correlation, researchers hypothesized because it just looked at very [00:10:00] different things.

Michele: Oh, that's interesting. Now, have you heard of the children's kitchen? I think it's called the children's kitchen task assessment.

Lacy: Uh huh. I feel like this is similar

Michele: It's yeah, it's similar. That one's a free one that you can get, um, online where you have them make Play Doh and then you rate them, but there's no standardized score that comes with that. Does this one have a standardized score?

Lacy: Yeah, this one does. Um, and it's norm referenced against typically developing kids, and there's some interesting data on some other populations.

Michele: Oh, okay. Nice. So how does it work?

Lacy: So students are given a printed calendar that covers one week, and then they're given between 10 to 18 appointments or activities for them to put on their calendar. This is where it can get tricky, because some of the appointments can overlap if a student isn't careful. For example, the list of appointments that they need to put on the calendar would be like, “You have a doctor appointment on Tuesday [00:11:00] at 4 p. m. for 60 minutes.” And then the next thing on the list might be “You need to study after school for 30 minutes, two days a week.” Now with that, obviously probably won't be able to schedule both of those on Tuesday. So a student would need to rearrange their calendar in order to fit. 

Michele: Oooo, that sounds tricky. I could see how the kids might, um, get tripped up with that 18. That's a lot. That'd really fill a calendar.

Lacy: And there's some extra special twists to it too that increase the cognitive demand even more. The student being assessed also has to tell the therapist when it has been seven minutes. Since the start of the assessment. So they're also keeping track of time.

Michele: I love that

Lacy: Mm

Michele: I need someone to administer this to me because I'm the worst at like predicting how long something's [00:12:00] taken or how long I've been working on something. So, ooh, that's cool.

Lacy: Yeah. And then there's another twist the therapist giving the assessment has to say, now focus, don't pay attention to any distractions, but then the therapist distracts the kid by asking like three random questions throughout and the kid's supposed to ignore them. So the little troublemaker in me just loves this part.

Michele: Sabotage are in you. I mean, like every pediatric therapist has to have a little bit of that in you because you're constantly trying to increase the challenge for the child. I am wondering who in the world could do this test successfully. It obviously isn't for younger kids. So I think you said middle school age children. What's the age range on it?

Lacy: Yeah, it has norms for ages 12 to 94 years old. And the manual says it can be used for individuals with a lot of different conditions. It lists a variety of serious mental health conditions, chronic [00:13:00] illness, thinking like long COVID, cancer, and also neurological injuries or illness, “at risk youth,” and individuals with developmental disorders. There's also three different formats or difficulty levels that you can choose to use. Based on the age and how much time you have to administer it.

Michele: That's good. Cause it sounds kind of complicated. Is this something that's been around for a while or is it newish? I mean, I've never heard of it, but it sounds pretty well established.

Lacy: Um, it's newer. It came out in 2015.

Michele: That shows how old we are that we think 2015 is still relatively new.

Lacy: Yeah, that's true. It's relatively newer.

Michele: After you have administered this and you've tripped up your kid in every way possible, what kind of information does it give you? I mean, I could see how it might tell you about their ability to plan, their prediction, their on task behavior, working memory. I mean, it [00:14:00] seems like you could get a lot of data from it.

Lacy: Mm hmm. Yeah, when you're scoring it, you get information about the number of errors that the person made. You're getting information based on the norms, how other people similar ages are scoring. But another part that I really like is that it asks the person being assessed to give a self report on how they think that they performed. And in that set of questions, it asks the person about their approach, like what was their thought process as they were filling it out. So it gives some good qualitative info and some good quantitative info.

Michele: Got it. I know at the beginning, we said the Weekly Calendar Planning Activity that this was both an assessment, but also that there was some sort of intervention involved. So we talked about the assessment part. What's the intervention?

Lacy: Yeah, so with the intervention, once you identify the strategies that the student used, you can build in some other strategies along the way. 

Michele: Does it give you a list of [00:15:00] strategies?

Lacy: Yeah, there's a list of strategies and you're checking off which ones they used along the way.

Michele: Because my executive function might need some support from a list of strategies to check off.

Lacy: yeah,

Michele: This is funny. We were, we, we have this test for working memory and it's on the iPad and it doesn't give you a answer key. So you just have to listen to see if the child got it right. And then if they do, you progress them. And if they don't, you stop. And so I have like, I had a student go through and do the test over and over again, cause there's different versions of it and make me an answer key because I could not keep track on whether or not they were getting it right or not. So, you know, we all have our strategies.

Lacy: That's funny. It's a lot to like keep in mind what's going on and observe the child at the same time. They should have an answer key.

Michele: They should have an answer key. I agree. This sounds like an interesting assessment, but I am kind of wondering who I could actually administer it to. It sounds hard, especially if I'm thinking about my kids with [00:16:00] autism. I mean, I know we're talking about middle schoolers, but did they talk about that at all?

Lacy: Yeah, the researchers wanted to expand what is known about executive function explore a new population with this assessment. So they looked at middle school age children, which is 11 to 14. And they were also looking specifically at students with autism spectrum disorder without an intellectual disability. And I think to this day, it's still the only article that is focused solely on autism and the Weekly Calendar Planning Assessment. With this group, students with autism without an intellectual disability, this assessment was really hard. Their scores were pretty low, and they also lacked self insight about their performance. Most of them said that they did really, really well, even though their scores didn't reflect that.

Michele: I love that for them.

Lacy: I know, I love the optimism.[00:17:00] 

Michele: I mean, I think, honestly, I just don't think it's a good match for the autism population. I can definitely see this for my ADHD kids, but I think the percentage of kids with autism that have also have an intellectual disability is really high. So I don't even know that I have that many kids that would fall into that area of being able to do it at all on the spectrum, but I can definitely see my ADHD kiddos, my kids that, you know, have just like language delays or developmental delays, kind of like a little bit more mild involvement where you're just trying to fine tune it in order to help them, especially in a school setting.

Lacy: Your thought was exactly what I was thinking too. I have an 11 year old son and I think this would be really hard for him. They're just not used to thinking about calendars and writing things down.

Michele: Mm hmm.

Lacy: But in spite of my skepticism, have given this assessment to typical kids that [00:18:00] are the same age and their results were really, really different.

Michele: Okay. So they proved us wrong. 11, 14 year olds can use a calendar and plan out their own appointments. Sounds like you're enabling your child, Lacy.

Lacy: Maybe, maybe.

Michele: Little helicopter parenting happening there. It happens to the best of us.

Lacy: Yes. The typically developing kids entered all 18 appointments on the calendar, whereas the students with autism entered only 8. 7 on average activities into the calendar. So the biggest gap was. The mismatch in location on the calendar or just omitting it in general. The researchers didn't really know why. They thought maybe the kids with autism just kind of got flustered and gave up or they just kind of rushed through it. The other difference was in the strategies that they used to approach the calendar. The typically developing group put in the fixed appointments first, so the ones that had to be in [00:19:00] certain spots and then approach the more flexible items afterwards. Whereas the group with ASD had some different strategies, they used their fingers and kept rereading and trying to think through things.

Michele: Interesting.

Lacy: Overall, the researchers thought it was a valuable tool and captured important information about executive function. So, while I was reading this article, I kept thinking of a really good friend of mine who has a 12 year old son with autism and no intellectual disability. And my friend really struggled with getting help for her son, specifically for executive function at the school. He had every textbook sign showing that he was having difficulty with executive function.

She sought out help from an outpatient OT and a school based OT, and they just kind of shrugged her off. Then in a meeting with the principal where she was trying to get an IEP or some kind of services, the principal said, quote, “We don't do IEPs or provide [00:20:00] services just because a student has a messy desk.” And that was the end of it.

Michele: Ooh. Brutal.

Lacy: I know. now I wish I could go back in time and tell her about the weekly calendar planning assessment. Maybe she could recommend that someone give it to her son. And I think it would have given her the data that she needed and the words to advocate for her son. And maybe it could have helped the principal and the team learn more about executive function

Michele: Yeah.

Lacy: Rather than only hearing “problems with a messy desk”.

Michele: Oh, that's so heartbreaking. I kind of think that executive function is like the new sensory integration where we're just really starting to understand how pervasive it is. And if you have problems in these areas, it's going to impact everything else. And so when you think about executive function, you think about inhibition and how you have to be able to turn off external distractions and internal distractions in order to focus on your work. And [00:21:00] you think about working memory and how you have to be able to hold on to these. Um, bits of information for like spelling, copying from the board, doing a math problem. You think about cognitive flexibility and that goes into all of the day and we have to do something else. Like it applies to every single thing at school and if you're struggling in it, Yeah, you're gonna have a messy desk like that. But that's the canary in the mine. I guess my question still remains for this article. We've identified this great assessment that could give good information that could have supported this child and maybe getting services to help them in the school district. But we still haven't talked about intervention. Like, what do we do once we know they have a problem? What in the world are we supposed to do about it?

Lacy: This article talked about using metacognitive strategies and task specific strategies. I was a little bit bummed because it just kind of stopped there. I

Michele: Oh, [00:22:00] so helpful. Thank you. I'll do a metacognitive strategy. Why didn't I think of that already?

Lacy: So I dug a little bit deeper and I have some answers for us. So specifically when looking at filling in a calendar, you could do what the typically developing kids tended to do, which was put in the fixed appointments first, and then plan the other activities around it. Other ideas would be underlining or highlighting keywords, verbally rehearsing instructions, crossing items off the list when you're done, and then double checking your own work when you're finished. So, very cognitive interventions.

Michele: Yeah, really just giving those are specific strategies for that skill of calendar planning. You know, we, um, dug deep in our clinic with the executive function stuff, because we were trying to do a research study on it. And I have so many resources for how. Intervention ideas for addressing executive function with a lot of [00:23:00] different types of kids, not just ones that can engage in metacognitive strategies. And in fact, none of those are included because we were really trying to look at things that are more accessible to kids that maybe are a little bit more involved and can't do that metacognitive thinking. So if anyone is interested. DM us, sign up for the newsletter. You can get a whole slew of additional information about how to address it. And I have a, I actually have a course that I teach on Summit Education on this specific. Intervention technique. Okay, Lacy. So you've shared this study with us. It's time to wrap it up with our final three questions that we always end with where we talk about the population, the key ingredients and the mechanism of action. So remind us the population of the study was...

Lacy: Middle school students age 11 to 14 years old with autism and without an intellectual disability.

Michele: Got it. And the key ingredients, which are [00:24:00] something that must be present if you're planning on implementing this with your clients in your session are…

Lacy: Knowing how to administer and interpret the Weekly Calendar Planning Activity, because this will inform your therapy goals and your intervention process. with compensatory strategies, effective organization strategies, some cognitive strategies, metacognitive strategies.

Michele: And then we also always talk about the mechanism of action, which is why the researchers think something is effective. What's your thoughts on that for this study?

Lacy: So we don't really have a mechanism of action because the article is more about an assessment rather than an intervention. But the findings suggest that OT intervention should incorporate planners, schedules, and step-by-step task management to improve executive function at school.

Michele: Yeah. And I would think also, you know, the reason those things work is because we know that [00:25:00] executive functions are something that you can improve. You're not just stuck with what you're born with. You can practice it. You can train it. And based on the theory of brain plasticity, we know that it's going to improve based on intervention. And that's it for our research review. We hit the highlights and we shared our own thoughts, but of course, for more details, pull the article and read it for yourself.

Lacy: And now it's time for People, Places, and Products, where we shine a spotlight on something that is making us happy. Michele, what do you have for us today?

Michele: I am excited to share about a very favorite toy that we use in our clinic. It's one of those toys that you know everybody loves because you can never find it. It has a place where it's supposed to live, but then everybody grabs it and uses it and we use it. The PTs use it. Speech therapists use it and OTs use it and they have it in the early intervention room and sometimes it's over in the neuro room and then I use it in [00:26:00] my sensory gym. So I just think it's a really adaptable and fun and motivating toy.

Lacy: Sounds like you need, like, two or three of them, then.

Michele: I know I have it in my Amazon cart right now to buy more for the clinic because it's ridiculous to have to be walking all around looking for it. So yeah, we definitely need more. But it's funny to me. I never would have tagged this particular toy as being our MVP. Like if you were to line up all the toys, I wouldn't have guessed. But for whatever reason, it seems like all children love the Melissa and Doug Scoop and Serve Ice Cream Counter. It's this little wooden box that has little ice cream cones in it and an ice cream scoop. And then it has different types of already pre made scoops of ice cream. It's all made of wood, but they have like little sprinkles and they have the chocolate mint ones. And then they have additional things that you can put on top of it. And [00:27:00] they clip together kind of like a Lego, you know, like they clipped together that way. And for whatever reason, every child seems to love this product. So put it in your Amazon cart along with me and try it out with your kids. Let us know how it goes.

Lacy: I love the Melissa and Doug stuff. I haven't tried the ice cream one though. I'm going to have to check it out too.

Michele: It makes me hungry every time we pull it out. Come on -ice cream's my weak spot. Let's not play with that this time.

Lacy: If you've got questions, we've got answers. Email us at ideas at play podcast at gmail. com or DM us on social media and we may include your question in a future episode. This week's question comes from Nicole in California. Nicole said, “I saw a post on Instagram where they said if you spin a kid to the left it stimulates the left cerebellum and the right cortex and this helps improve anxiety, sensory [00:28:00] issues, and attention. And then when we spin the child to the right we can improve dyslexia, language, and learning difficulties. Is this true?” 

Michele: Lacy, I wish this were true. Wouldn't that be so simple? Our jobs would be just done. Problem solved. Just put the child down, spit him around. Got a little anxiety? Go to the right. Got a little language problem? Go to the left. And let's just get this job done. I think this highlights how important it is that when you're looking at ideas on the gram that you were considering the source and making sure that, you know, that that source is a solid person who has some experience, who has some credibility, because this is one of those quasi science things where it's sort of true. It's actually very interesting when you get into the, weeds looking into the research about vestibular stimulation and how it impacts the brain and how it could impact other [00:29:00] areas of functioning. But I guess this is the problem with social media, right? You have to be able to make it into something short and quick and catchy and and people tend to overstate things for that reason. And that's what's happening here. But yeah, definitely when we engage the vestibular system. We are stimulating the cerebellum. And the cerebellum is so interesting because the old school mindset was that the cerebellum is useful for motor control and coordination, which is still accurate. I think that's what everybody thinks of when they think of the cerebellum, but newer research has shown that has prolific links all throughout the cortex. And it does link to the limbic system, for example, and that's why we know if we get on a swing and we do some nice, steady, linear swinging, we are going to have an impact on arousal, and it's going to bring them into a calmer state, or if we do erratic and rotary input, that that's going to help to alert them and wake up their nervous system.[00:30:00] And so that actually is true. It can help improve anxiety or sensory issues or attention, but it's not so straightforward where it's like a recipe, just spin them to the right 10 times and you will solve their anxiety. I mean, that would be wonderful if it was true, but it's not. But yeah, definitely the cerebellum does it links to the limbic system. It links to our sensory motor system. It does link to some areas that have to do with language. So, I mean, it's not like totally out of the blue, but it's not exactly accurate.

Lacy: Very overly simplified.

Michele: Yeah, exactly. And that is what you find oftentimes on social media, except for not on our social media. So if you want accurate information, check us out on Instagram under @ideasatplay, and you will see just the truth, only the truth and nothing but the truth.

Lacy: Love it!  This week we reviewed the weekly calendar planning assessment to see how we can measure executive functioning [00:31:00] development. Then we discussed how that info can be used to support children in the classroom.

Michele: Thanks for listening to Ideas at Play. Did you know that most people find new podcasts based on recommendations from friends? It would really mean so much to us if you shared this episode with a friend who could also enjoy it, and we can all be evidence-based occupational therapists. And if you want more ideas for your sessions, you can find us on Instagram under Ideas At Play, or email us at IdeasatPlayPodcast@gmail.com to ask us a question or get our newsletter.

Lacy: Until next time. Stay informed, stay curious and stay playful.

Out take

Lacy: Spin to the left, spin to the right, and now you're fixed. cha real smooth.

Michele: I think it's a hit. I think you just created a hit.