Milliseconds Matter: Understanding How We Read with Eye-Tracking Research

Ryan Buggy of The Reading League is a PhD student in cognitive psychology, and is currently using eye tracking technology to study how the brain processes language. Ryan discusses the cognitive processes that impact reading and language and what the latest eye-tracking research can tell us about how we read. Based on this research, Ryan also shares best practices for educators and practitioners. 

Mentioned in the episode:

The Road to Running Records

The Reading League Journal: Measure What Matters

ACT: Reading Between the Lines

Transcript

Lauren Clouser:

Welcome to the LDA Podcast, a series by the Learning Disabilities Association of America. Our podcast is dedicated to exploring topics of interest to educators, individuals with learning disabilities, parents, and professionals to work towards our goal of creating a more equitable world. Hi, everyone, welcome to the LDA Podcast. I'm here today with the Reading League's Professional Learning Director for Research and Impact, and a PhD student in cognitive psychology at UMass Amherst, Ryan Buggy. Ryan, thanks so much for being here today.

Ryan Buggy:

Thank you for inviting me. Happy to be here, of course.

Lauren Clouser:

We can't wait to talk to you. So I wanted to give you a chance to tell us a bit about yourself, your educational background, and your research focus.

Ryan Buggy:

Sure. So right after undergrad, I majored in psychology in undergrad, and I was always really interested in education, especially since my younger sister is special needs. So I always saw the impact that education can have on people and the amazing teachers she had, and I thought that education would be a great place if you really like psychology and cognitive science, education is a great way to apply those skills. So I taught high school English, and then after that I started working in a curriculum department at a school district in Massachusetts. And specifically I was helping out with data and assessments. And I think that was kind of the first sort of clue that I had that there were areas where maybe some of the findings from cognitive science hadn't fully been translated into the education world. So in the different assessments we would look at, there'd be times that I would just be a little bit confused. I assumed that everything we were told to do must be correct, it must be coming from an evidence-based perspective.

But I would just be confused sometimes about the way we would measure certain skills and what assumptions we were making about how skills would develop, especially in reading. So that got me really curious about what is the evidence base around reading. And at the same time, my school district had just started working with the Reading League's professional learning team to learn more about literacy instruction. So once they came to our district, that really opened my eyes and I found out that there was a ton of cognitive science about reading and literacy development. So I really latched onto that and was so excited to learn about it. And after a couple years, I ended up moving to work for the Reading League. So I've been with TRL for almost four years on their professional learning team. And then separately from that, I'm i my kind of full time role now is that I'm a PhD student in cognitive psychology at UMass Amherst.

Lauren Clouser:

That's fantastic. And we can't wait to talk to you a little bit more about your research now.

Ryan Buggy:

Sure. Yeah. So at UMass, I'm in the UMass eye tracking lab. So there are a lot of folks who study reading by using these really fancy cameras that track your eyes, and they can take a new image every millisecond. So a thousand times a second to see exactly where the eye is fixating as readers read sentences or longer chunks of text. And we use that information about how long words are fixated and where the eyes go as they read to try to better understand how language is processed in the brain. So a lot of that reading research actually came from UMass.

There is a psychologist named Keith Rayner who developed a lot of these techniques in the 1970s and 80s at UMass, and the lab has existed continuously since then. Right now, Adrian Staub is the director of the UMass Eye Tracking Lab, and he's my PhD advisor.

Lauren Clouser:

That's great. Well, and then we'll dive into that a little bit further. And I want to connect it to the science of reading, which is really big right now. It's not new, but we're hearing a lot more about it in the media. And it's focused on phonological awareness and its relationship to phonics and decoding. And your work is focused a lot on how oral language skills and syntax contribute to becoming an efficient reader. And you use the eye tracking research that you just mentioned to investigate this. So what are the most relevant aspects of that research for educators and practitioners?

Ryan Buggy:

Sure. So before I even get into the oral language and syntax component, I do just want to say sometimes I think that we talk about reading. We talk about phonics and phonological awareness on one side, and then we talk about language and vocabulary on another side. And it makes sense from this simple view of reading perspective. But sometimes I think that we might have the perspective that once we have bought a phonics curriculum or started teaching word recognition skills explicitly, then that's all done and solved and we don't have to worry about word recognition. So I do just want to acknowledge that I've been to a lot of schools that are doing fantastic work, but I don't think that we should think of word recognition skills and phonemic awareness and phonics as being done and solved. There's still a lot of work that we can do there. But you're absolutely right that oral language and especially syntax and sentence structures are more what I study.

And we have a lot of research. I mean, if we think about reading, I really love the Snowling and Hulme model that reading is language. So what are we reading? We are reading language, and our language skills are really what influence our reading comprehension, our proficiency and fluency as readers. So one thing that I look at is how do we parse sentences, and what are the different linguistic factors that can affect how easy or difficult it is for us to read sentences? So in the lab, one way we could study that experimentally is by having different sentences that only vary very slightly, maybe in a certain critical word, that could be made maybe more or less predictable, or maybe some sentences have one kind of clause and other sentences have a different kind of clause. What we look at is how that changes readers, the amount of time that they spend fixated in a certain area of that sentence. And to be clear, too, these are very, very minor differences that we notice in the eye movement record. So we might see a difference of 100 milliseconds, like a tenth of a second, and that would be considered a really big effect. So it's not something that you can really get from a pen and paper assessment.

If I think with my teacher hat on, if I'm thinking about how I might evaluate students' language comprehension, I might turn toward multiple choice comprehension assessment or some sort of other assessment where students are very consciously deliberately crafting a response or responding to a test. In the lab, a lot of what we are studying, we assume that these effects are unconscious and automatic. So we might only notice these effects if we bring in 100 subjects and have them do many, many trials of the same experiment. And then we can detect just this really tiny difference in reading times. But that actually translates to a pretty big effect in terms of the cognitive processes that underlie this language comprehension.

Lauren Clouser:

Absolutely. And that's so interesting. And you said this technology's been around since the 70s, too.

Ryan Buggy:

Yeah, and it's changed a lot, too. So now when you look at these eye tracking cameras, they look almost like a little box that's sitting on the desk. They're pretty portable, so you could do studies not just in a lab. But decades ago, they had to really make sure that your head did not move at all. So they actually had this technology where they had a kind of gel like they use in a dentist's office.

So readers would have to bite down on this bar that would kind of get harder so that it sticks to exactly the form of your teeth so that you couldn't move your head at all while you're doing this experiment. So that research has changed a ton in the last few decades in how the technology has advanced, but the same general idea remains the same, that we can tell a lot about how reading works just by looking at readers' eye movements as they progress through text.

Lauren Clouser:

Wow. That’s probably a lot more comfortable now to do.

Ryan Buggy:

Definitely. I'm happy I'm not responsible for that now.

Lauren Clouser:

I can't blame you! So in your article, the Science of Syntax and Predictability, you note that while skilled readers make predictions often while reading, you refute Ken Goodman's contention that readers are consciously using syntax and semantic clues while reading. And then the three cueing system is unfortunately still a big part of early reading instruction. So can you discuss what the eye tracking research indicates about three cueing and maybe just to start us off, tell us what three cueing is for those who might not know?

Ryan Buggy:

Definitely. Ken Goodman was a linguist and an educator and he had a lot of theories about how reading might work. And I think that one of the big driving principles behind his work was just noting how fast we can read. It's pretty astounding if you think about this as an ability that we didn't evolve to do, and yet skilled readers can read text so expertly and without thinking. And he had this idea that it didn't seem feasible that we could actually be processing every individual letter and every single word. That just seems so incredible that it can't possibly be true. And so he had these theories that maybe what readers do is something like taking a little sample of text so our eyes would dart around the page and we would look at certain words and maybe we would actually decode those words. But most of what would happen would be that we are filling in the blanks in a sort of top down process.

We are using our language skills and in our knowledge of the world and knowledge of vocabulary to make really good predictions about what the text likely says so that we don't have to effortfully decode every single word. And so the idea itself, I think it's not completely irrational. Reading is an astounding process and it seems amazing that we could possibly become so expert at this. And how could it be the case? Especially when you look at early readers and you see how difficult it is. So I see where the logic comes from. But when he was making these theories, he didn't necessarily have access to the kind of scientific research around reading. Eye tracking the way it looks today was very much in its infancy.

And we didn't have other technologies to study reading as carefully. So a lot of his theories were not directly tested at the time. But as we started testing these theories, people really noticed that reading didn't work exactly the way that Goodman thought it did. So he came up with this three cueing theory where he thought that there were three different cues that readers relied upon to read a word. So one of them might be that we make guesses about what we think a word might mean. We use our knowledge of vocabulary and meaning to decode a word without having to actually process it. Another idea is that maybe readers use their syntax abilities. So maybe if a reader says ‘the’ instead of ‘a,’ then that's because they were correct that there was a determiner and that we expect an article in that position.

They just guessed the wrong article. And then he also did attribute a role to what he called visual or graphophonic cues. So actually using the letters in a word. So some of the early research, some of this isn't even eye tracking research per se, but one of my favorite studies, two psychologists, Keith Stanovich and Richard West, they wanted to see, is it really the case that readers use context to decode a word? So they just took pairs of sentences. One sentence might be ‘the pilot flew the plane,’ and the other sentence might be ‘the dog buried the bone.’ And you just take the last word in those sentences and swap them. So now you get a sentence like ‘the pilot flew the bone’ and ‘the dog buried the plane.’

So a sentence where that final word is really not something you would guess from context. It's an incongruent sentence frame is what they called it. And they just had readers read these sentences, and they look at how long it took them to read those sentences and how accurate they were. And there are a lot of really interesting findings in their paper. But to me, one of the big takeaways is even in that incongruent condition where you're reading a sentence like the dog buried the plane, readers are still more than 95% accurate. And it might take them a little bit longer to read that sentence. But we're talking on the order of milliseconds, tiny fractions of a second, not anything that you would even notice just by observing the reader. So that's some pretty compelling evidence that maybe readers don't actually use context when they're trying to decode words.

There's another study that Jenkins and colleagues did where they took folktales that would be at an appropriate level of difficulty for those students. And they scrambled all the words in the folktale. So it just looked like a random list of words that had no sort of meaningful context. And yet, readers who score the highest on comprehension assessments also were able to read those random lists of words the most accurately and the most automatically. So that's another connection that, okay, it seems like really skilled readers can read words fluently and accurately. And when we move into the eye tracking research, we see that readers fixate almost every single word of text. We actually are processing almost every word.

And we have some really interesting evidence that those words that are skipped, because it is true that sometimes we skip over a word when we read, but those are overwhelmingly very short, very high frequency words that actually can be recognized in what we call the parafoveal preview. When you're looking at a word, you see that word itself very clearly. But you also see in what's called your parafoveal vision, peripheral vision, just a little bit to the right or left of fixation, you can get some information about the upcoming word, like the next word right after the one you're looking at. What we see is that expert readers are actually so good at decoding that they decode the word they're fixating on, and then they can even begin processing the subsequent word before they've even moved their eyes. Because we can decode so fast, we can decode that word faster than we can even trigger an eye movement. So sometimes what happens, especially for those really short, high frequency words, we actually fully recognize that word before we've even fixated on it. And so then we're able to kind of reprogram our eye movement to skip over that word and read the next one. And so there are a lot of paradigms that have looked at this.

Keith Rayner is the one who sort of first looked at these effects, and he did so with this kind of fancy paradigm where basically you have a computer screen, where as readers read just in the area where they are fixating, they see that word clearly, but all the other text is degraded so that it just looks like the letter X over and over and over again. So you can't get any parafoveal information, just specifically what you're looking at, and what he began to notice as he manipulated the size of how much text readers could get. So he would make it so that maybe you get the word plus the next couple letters and then plus a couple more and so on, as long as readers get about 15 letters to the right of fixation or reading proceeds as normal. But if it gets any smaller than that, reading slows down dramatically. So he really took that as evidence that readers need to fixate on almost every word in text. And yes, sometimes we do get a parafoveal preview, but it's nothing like random sampling along a page that Goodman predicted. It's very systematic, and we're very sensitive to those specific words and their spelling.

And we just get faster because we do become experts at this task.

Lauren Clouser:

That's so interesting, and it's so exciting, too, that we're able to track these, like you said, very almost microscopic, easy to miss movements that can mean so much and can really provide us with so much information. So that has to be really exciting to be a part of. 

Ryan Buggy:

It is really exciting. It's exciting to see that history, too. I just feel very fortunate. My advisor was Keith Rayner's student, and he was doing this research in the very lab where I get to be a student now. And so I just feel very grateful to get that opportunity.

Lauren Clouser:

Absolutely. And I want to talk a little bit more about how this applies to practice then, because you did bring out that there is that research to practice gap. So the takeaway in that same article is that teachers can support language development throughout school years by modeling complex language and providing opportunities to provide syntax in rich linguistic environments. So what would that look like in a classroom?

Ryan Buggy:

Yeah, so I think it's hard to separate sometimes what is teaching syntax versus what is teaching other aspects of language? And I actually think maybe it's not important to make that distinction. So much of this is intertwined. And so when I think about what is really rich, high quality language instruction, a lot of that is thinking of language almost like you were putting money in the bank for kids. Our brains get so good at mapping this vocabulary about making predictions about upcoming language that we see both in reading and in speech, that readers are unconsciously making predictions about what comes next and using that to kind of inform our parsing of sentences. And so the way that we get people to have all of that linguistic data is by exposing them and giving them a lot of opportunities to both listen to and produce oral language. And so this is an area too, where if we think about phonics and phonemic awareness, like the big message, there is explicit, systematic, direct instruction.

When we think about language comprehension, it's a little bit different. So there is a place for that very structured language instruction. But we are also really good at just kind of soaking up language and so even just before kids ever enter school, we can see a relationship between the amount of language that their caregivers use and how many back and forth interactions caregivers have with their children, and those children's eventual language development. So it really starts even before school with having these back and forth interactions with your kid where they're saying something and you're expanding, kind of reflecting what they say and adding a little bit to it. And these back and forth interactions, or you're narrating your own thinking as you walk about the house. And we know that your baby can't even speak yet, but just hearing you giving them all of that rich linguistic information is already starting to tune their brain. Once we get to school, then I think we should think about what is the focus of a given lesson. So some of the day, if we are focusing on word recognition, we're going to use words and texts that are easy enough for students to practice decoding using their existing phonics abilities.

But another part of the day, if my focus is language comprehension, I'm going to choose a text that maybe is something they wouldn't be able to read independently. Like, I can be reading a young adult book to these kids who maybe are still learning how to decode, but the reason why I'm reading it out loud to them is to give them that really rich vocabulary and expose them to these syntactic structures that aren't as common in oral language. So in younger grades, that may look like reading aloud and talking about the new words that we're introducing them to and talking about those sentences we're reading in the upper grades. As kids become better and better at word recognition, then I think it's drawing their attention to features of sentences, drawing their attention to the punctuation that's being used, to the way clauses are adding some meaning about the subject, to drawing their attention to what is the action here? Who is doing something? What are they doing? How are they doing it? Where does each part of the sentence answer that question for us, encouraging that kind of metalinguistic thinking. I think the big takeaway is that all of this language exposure is really helpful for kids, but we want to be really mindful about the quality of it, too, and relying on things like children's books, which just have a greater proportion of really rich language versus kind of more informal oral language.

Lauren Clouser:

Yeah, that's a great point to bring up. I'm so glad you did. So another common literacy practice that you want to tackle is the use of running records. Could you tell us a bit more about what are running records, and why are they not a sound assessment practice?

Ryan Buggy:

Yeah, definitely. So running records came about kind of hand in hand with the three cueing system. And there's a great article for anyone who's interested by Annie Unger, Tanya Serry, and some of their colleagues at Latrobe called the Road to Running Records. I really love that article because it kind of traces the history of how the three cueing system is very much intertwined with running records. So Marie Clay, at the time she was studying reading development, was very much influenced and in the same intellectual territory, I would say, as Ken Goodman. And so she would look at the kinds of mistakes that kids would make as they're reading out loud. And she ended up kind of coding these mistakes using MSV, meaning, syntax and visual cues, to kind of understand what cues did kids seem to be using correctly as they read. And then if they made a mistake, could you maybe suppose that that mistake is a result of them not using one of those cues correctly? So, if a kid is reading a sentence and the sentence says, I like to eat chocolate, and the kid says, I like to eat candy, maybe the kid got the meaning close enough, but they didn't look at all the letters carefully, or maybe they didn't look at a picture, something like that.

And so the issue with these running records, that they are a problem for the same reason that the 3 cueing system is invalid. I have a kind of really nerdy way to put this, but I think about a movie that I watched with my dad growing up. I don't know if you've ever seen Monty Python's Holy Grail, but there's a really funny and kind of ridiculous scene in it where they're trying to decide if they think a woman is a witch. So they make this whole pseudological deduction where they say, okay, well, if she's a witch, she must be made out of wood. And if she's made out of wood, she must be light enough to float. And if she's light enough to float, she must weigh as much as a duck. So they put her on a balance scale where she's on one side and the duck is on the other, and they say, okay, therefore she weighs the same as a duck, she must be a witch. And that's like a ridiculous scene. And we laugh at it. But I think it illustrates the point here that if our theory is wrong, then any test that we designed based on that theory, that's going to be wrong too. 

So if it is not the case that kids rely on the three cueing system to read words accurately. We can't really rely on a test that's designed to look at each of these three cues and tell us which one kids are using, because good readers aren't relying on meaning and syntax and visual cues to read. They're just looking at the letters. There are a lot of issues if we talk about some of the technical properties of these assessments. One big issue is that they have very low reliability. What that means, if we think about if you take the SATs, let's say if you take it once and then you take it again, and you take it again the next day, you might get very slightly different scores.

And that doesn't mean that you got smarter one day and then you forgot some stuff the next day. That just means that not every kind of measurement instrument is going to be perfect. When we're trying to capture these kinds of interior mental properties of someone, our assessments are not going to be perfect. But assessments vary pretty tremendously about how reliable they are from one day to another. So some assessments, like I'm thinking about, like DIBELS and Oral Reading Fluency and Acadience and these products, they have quite high reliability where the correlation between how well you perform one day and how well you perform the next day might be 0.8 or 0.9 or higher. For these running records, the reliability is so low you'd have to administer them again and again and again about nine or ten times to actually get a sort of stable measurement of the student's reading ability. So think about that in the classroom. If it took me maybe 40 minutes to administer a running record and code all the cues and self corrections and so on, you're telling me I have to give that 10 times to get a reliable measurement of the student? It's just not practical.

Another issue is that we can't really be sure that the mistakes students are making are always going to be systematic and easy to interpret reflections of what is actually going on as they read. So maybe that kid read chocolate as candy. Maybe because they were looking at a picture, maybe because they didn't know how to read that word and so they just made a guess. Maybe they were thinking about candy and that just came out of their mouth. There's any number of reasons. And I can't really be sure that I have correctly identified the right reason why kids make a mistake. In that article I mentioned by Unger and their colleagues, they make this really good analogy where they say focusing on the errors kids make to learn about skilled reading is like watching someone fall off a bicycle to learn about skilled cycling. The errors represent times when that process has failed.

That's not a time when that process is going smoothly. So we don't want to draw teachers' attention to these errors and try to link them to the three cueing system. We really want to focus on what we know about how reading develops, and that's through accurate and automatic word recognition and through oral language development.

Lauren Clouser:

Definitely. Those are all such great points. And to that point, what are some better assessment practices the teachers could be using? What are some more factors that we should be considering when we're looking at assessment?

Ryan Buggy:

Sure. So there are a few assessments that I really think about. And one thing that I would encourage teachers to ask is, what is the purpose of this assessment? Do I know if it's valid, meaning, does it really measure the thing that I think it's measuring? Is it reliable? Meaning, am I likely to get consistent data? And then finally, is it efficient? There are a lot of ways that we can measure the same skill, and they are going to vary tremendously in their efficiency. What I think of as kind of the gold standard for valid, reliable, efficient assessments are a type of assessment called curriculum based measures, or CBMs. So those look like oral reading fluency or nonsense word reading or phoneme segmentation. There are a lot of different measures depending on where the child is in their development. And but the gist of these assessments, most of them have the child complete one task for about a minute.

So they're reading out loud for a minute. And then you score how accurately you score the rates of their accuracy and how many words they read correctly in that time. And it turns out that that measure, that rate and accuracy is very highly correlated with things like reading comprehension, with your future reading ability. Just in one minute, you're able to get a lot of information about this child's proficiency. Thinking about oral reading fluency, for instance, the correlation between oral reading fluency and reading comprehension in these lower grades is as high as 0.9, which is pretty astounding if you consider these comprehension measures. They might take an hour or two hours for kids to complete. And yet just listening to that kid read aloud for one minute and noting how many words they read correctly, that gives me a really good sense of whether or not this kid is developing as usual or whether this kid needs some more instructional support. That's a very, very efficient assessment.

There are also other assessments too, that look more at oral language skills. And I would just say in full transparency, measuring language and language comprehension is going to be a little bit harder than measuring word recognition. Word recognition is a really finite skill. It's sort of that if I'm thinking about letters and letter sounds, either you recognize them and you know all of them. And once you do, that's something easy to measure. But how do I measure language? Like, what vocabulary should I assess? What sentence structures do I assess? How do I know whether a difference in language represents a real difference in proficiency versus a difference in cultural linguistic background? But there are some fantastic researchers who have looked into these issues. I'm thinking about Tiffany Hogan in particular, as someone who I really admire for her work. And something that we might want to look at in the domain of language is specifically developmental language disorder.

This is a language disorder that occurs, depending on how you measure, at roughly the same rate as dyslexia. But I would just say my experience in schools is that I feel like when I work with teachers, everyone, or the vast majority of people have at least heard of dyslexia and know something about dyslexia. I would say that developmental language disorder, or DLD, there's a lot less awareness about this condition, and yet it occurs at roughly the same rate when we are measuring language. We might want to choose screeners for developmental language disorder to be sure that we've identified those kids with DLD and that we're able to provide them with the intervention supports that they need. Tiffany Hogan and her colleagues put together this fantastic list of DLD screeners that differ a little bit in terms of what age have they been validated with, what population have they been validated with, whether that might be native English speakers or English learners or speakers of a different language variation. Also, who is administering it? Can it be given by a teacher? Does it need to be given by an SLP, a school psychologist, and so on? It's not a one size fits all kind of…No assessment is going to work for every kid in every population.

But I really love their research and showing what are the screeners that we can select that's right for the kids who we have in front of us, so we can make sure that if there are kids with developmental language disorder, we've identified those kids so we could give them the support that they need. I might also recommend that folks check out, there's an article in the Reading League journal called Measure what Matters by Jessica Tost and her colleagues. It's free on our website. And two of my colleagues at the Reading League, they also hosted a webinar about this article. Andrea Setmaier, Elizabeth Lamoreaux, they put together this great webinar on the article. And we're also going to have what we call our summit, which is this two day event in Syracuse, May 5th and 6th, where we are inviting top researchers from around the country and also teachers and educators to collaborate and talk about assessment, look at some real assessment data, learn about different aspects of reading assessment from the teacher perspective, all the way through the district perspective and state level policy. So if you want to talk more about assessment and valid, reliable assessments, I really would encourage you to look into the summit.

Lauren Clouser:

That's fantastic. Thank you for sharing that. It's so exciting that we're having these conversations because they're so needed. And I also wanted to thank you for bringing up developmental language disorder. We actually just recently talked with somebody who was discussing the similarities and differences between dyslexia and DLD and just how a lot of people aren't aware of how much language impacts reading. You think reading, you think dyslexia, and really the two are way more intertwined than most people are aware of. And it goes back to what you said earlier as well, that oral language is often more simple than reading. So your suggestion to read older level books to younger children, to expose them to that language are all just such great tie-ins.

So I wanted to give you a chance to talk a little bit more about that if you'd like to sort of dispel that myth that reading is just dyslexia.

Ryan Buggy:

Yeah, I mean, people differ pretty tremendously in their reading ability. We know that. And I think that sometimes it's tempting to think, okay, if I have this third grader, I'll say, especially talking about third graders, because there's sometimes a transition that happens where I think we assume that people should be proficient readers by now. And sometimes that happens in upper elementary. And so if you have a third grader who maybe you gave them the state test and they did poorly, maybe there's this temptation to assume, okay, let's give them a phonics diagnostic and let's figure out what words they can't read and provide them with a phonics and phonemic awareness intervention. And a lot of the time that is needed. I'm not at all trying to say that kids suddenly magically become proficient in word recognition just because they hit third grade. Like, definitely a lot of kids do struggle with word recognition but we should also consider that there are a lot of other reasons why kids might have difficulty on a comprehension assessment, and some of that is going to be the language that they might not have.

There was this really great research by the ACTs, which I don't know where you're from, geographically. Some people do SATs, some do ACTs, but the ACTs, they put together this report that was called Reading Between The Lines. What they were looking at was what factors make comprehension easier or more difficult for these high school students who were taking the ACTs. And what they tried to do is disaggregate the different questions on the ACTs in terms of something almost like what standard they were aligned with. They weren't using that language in particular, but they were talking about is this question about the main idea of the text or about the author's purpose or so on. Interestingly, they did not find that what kind of standard a question was aligned with had any impact on student scores on those questions. Students who were generally getting high scores on the ACT could answer all those questions basically equivalently. And kids who were getting worse scores in the ACTs had equal difficulty with all types of those questions.

The same was also true for questions that were coded as literal questions versus those that were inferential. So what that makes me think about is we might be really fixated on looking at these standards and trying to think that we can boil comprehension down into just a set of types of skills that folks might have when really comprehension is so interconnected with our language skill as a whole and the background knowledge we have about a subject, our vocabulary and so on. So we should really be focusing on language instruction and what opportunities we create in the classroom to expose kids to rich oral language and help them produce rich oral language and increase their knowledge about the world and the topics you're reading about. All of those really matter. We shouldn't be fixated too much on, this is week one of my curriculum and it says on week one, I'm doing standard 8.1, and that's what I have to focus on. We're just reading a bunch of random paragraphs and focusing on the main idea, that is not likely to be helpful and translate to meaningful gains in students' language ability and their reading comprehension.

Lauren Clouser:

Yeah, I'm so glad you brought that up. And before I let you go, Ryan, I just wanted to ask if there's anything that you wanted to share about the future that you hope to do either in your PhD program with the Reading League, are there areas of research that you'd really like to get into?

Ryan Buggy:

Yeah, I mean, right now I'm really interested in studying morphological processing. I think morphology is really a hot topic in the instructional world. So it's something I want to study with eye tracking. But I would just say kind of more broadly, I think that we talk a lot about connecting research to practice, connecting educators to researchers. And sometimes I think in the way that we talk about it, especially in education circles, is kind of the onus is on educators for us to learn more about research and start reading these scientific journals and whatever. And I think that really, we should make sure that the responsibility is also on researchers. So I think that's sort of my ulterior motive here in doing this PhD program. I'm the only one, I think, in my program that has an interest in education.

Everyone there is a basic researcher who's really interested in the research itself, which is great. But I also am trying to encourage folks when I go to conferences and when I get the opportunity to meet these researchers, telling them, hey, do you know about the Reading League and do you know about these other organizations that are doing work in education? They would be really excited to hear from you and hear about your research. Because I don't think researchers always are aware of the fact that some of the research they do actually has pedagogical implications, or they think that somehow that research makes its way to schools of ed, kind of via osmosis somehow, but trying to encourage them. Like, we should be going to these conferences. We should reach out to educators. It's on us, too. So if we want that pipeline to exist, I think we have to build it in both directions.

Lauren Clouser:

Absolutely. I love that answer. Ryan, thank you so much for being here with us today and for sharing all of this exciting research. Let everybody know to check out the amazing work that the Reading League is doing as well. But, Ryan, thank you again so much for your time. It was a pleasure to talk with you.

Ryan Buggy:

Thank you so much. Thanks for having me. It was great speaking with you.

Lauren Clouser:

Thank you for listening to the LDA podcast. To learn more about LDA and to get valuable resources and support, visit LDAAmerica.org.