IJLCD - Outcomes from a community speech and language therapy service treatment waiting list

What happens to children on a waiting list when they aren't receiving treatment? How many continue to need intervention and how many improve spontaneously? It's difficult to research because it's too hard to collect this data ethically and also without impacting the subjects and therefore potentially changing the results.

Nevertheless, occasional opportunities arise for reviewing data from waiting lists. In this podcast we chat with Elizabeth and Sue about their research into an ethnically diverse community paediatric speech and language therapy service waiting list in the UK, where levels of social disadvantage are high.


The paper is:  

Outcomes from a community speech and language therapy service treatment waiting list: The natural history of 525 children with identified speech and language needs
https://onlinelibrary.wiley.com/doi/10.1111/1460-6984.12877

Elizabeth Hesketh, Paul White, Doug Simkiss & Sue Roulstone

First Published:
15 May 2023



Useful resources:
Outcome measurement overview: https://www.rcslt.org/speech-and-language-therapy/guidance-for-delivering-slt-services/outcome-measurement/#section-2


RCSLT Online Outcome Tool ('ROOT'): https://rcslt-root.org


Interview with Professor Pam Enderby:  https://soundcloud.com/rcslt/rcslt-podcast-pam_enderby-31-07-2020


Find out more about ROOT: https://www.rcslt.org/speech-and-language-therapy/guidance-for-delivering-slt-services/outcome-measurement/#section-2


Enderby, P. and John, A. (2019) Therapy Outcome Measure User Guide. Croydon: J& R Press Ltd


Language growth in children with heterogeneous language disorders: a population study, Courtenay Frazier Norbury , George Vamvakas,  Debbie Gooch, Gillian Baird, Tony Charman , Emily Simonoff, Andrew Pickles
Child Psychol Psychiatry; 2017 Oct;58 (10) :109 2-1105., https://acamh.onlinelibrary.wiley.com/doi/full/10.1111/jcpp.12793


PROTOCOL: Language interventions for improving oral language outcomes in children with neurodevelopmental disorders: A systematic review - Nordahl‐Hansen - 2019 - Campbell Systematic Reviews - Wiley Online Library https://onlinelibrary.wiley.com/doi/full/10.1002/cl2.1062


NOTES:
For RCSLT members, access this paper by navigating to the IJLCD website from our A-Z journals list here. Also, if you would like further information on the research terms used in the podcast, or many other aspects of research design, please navigate to the ‘Sage Research Methods’ collection from the Research Methods page of the RCSLT website’.


The interview is conducted by Jacques Strauss, freelance producer, on behalf of The Royal College of Speech and Language Therapists.

Please be aware that the views expressed are those of the guests and not the RCSLT.

Transcript

Transcript Name: 

IJLCD outcomes community therapy waiting list

Transcript Date: 

13 February 2024 

Speaker Key (delete/anonymise if not required): 

HOST:                         JACQUES STRAUSS 

ELIZABETH:               ELIZABETH HESKETH 

SUE:                            SUE ROULSTONE 

MUSIC PLAYS: 0:00:00-0:00:08

HOST:                         0:00:09 Welcome to the RCSLT podcast. My name is Jacques Strauss. Today is an IJLCD edition in which we interview authors of papers and the International Journal of Language and Communication Disorders about research that we think would be of interest to speech and language therapists in the UK and beyond. 

The title of today’s article is ‘Outcomes from a community speech and language therapy service treatment waiting list: The natural history of 525 children with identified speech and language needs’, written by Elizabeth Hesketh, Paul White, Doug Simkiss, and Sue Roulstone. 

In this episode, we were joined by Elizabeth and Sue, and I started by asking Elizabeth to introduce herself and explain how this research came about. 

ELIZABETH:               0:00:53 Hello, my name is Elizabeth Hesketh, and I’m a retired speech and language therapist who worked for many years as a clinical lead in Birmingham, which is a particularly large and ethnically diverse UK city with high levels of deprivation. I’d like to start out just by giving some background about how this natural history study came about. 

It happened entirely opportunistically, with the data used in the service collected as part of our usual service delivery. It is rather unusual to be publishing findings in this journal from a study that wasn’t set up with the usual experimental controls. And so, I would like to explain why I feel that our findings are reliable. 

Winding things back a few years, my colleagues and I carried out some case rating exercises to look at our levels of consensus when we diagnosed and selected children for the caseload. We found there was plenty of room for improvement. As a service, we developed clinical guidelines. We agreed protocols for formal and informal assessment. We agreed methods of recording our assessments, and we developed and evaluated a case selection and discharge tool, which included the use of Pamela Enderby’s therapy outcome measures impairment scale, which is quite a mouthful to say, so I shall now refer to it as TOM impairment scale. 

Also, very importantly, as it turned out, we designed a sheet to go at the front of case notes to summarise all this client data we were collecting. 

We then put ourselves through a period of whole service consensus building and reliability training. This was pretty testing, and at times quite threatening for myself and colleagues. We weren’t used to really deeply questioning each other’s decisions. But we came out of it the other end having found the process worthwhile, and up to a point even quite enjoyable. Our efforts paid off; we found that we had achieved, and over time we did maintain a good level of interrater reliability, and good agreement in terms of the cases we selected for the caseload. 

Then, staff shortages really began to bite and the service built up a particularly long waiting list of children who’d been selected for input after initial assessment. Time went on, things weren’t improving. Our NHS Trust funded us to review our waiting list. And I realised at that point, with some excitement, that we could do something positive, that something positive could come out of the situation, in that we had initial assessment data that could be gathered in pretty easily, and we had an agreed procedure that we would be repeating for our follow up assessments. 

In effect, I realised we had a mine of clinical and demographic information about these children who had waited, and this would allow us to check out how they had changed over time. 

With help from Doug Simkiss, the medical director of our division at that time and a co-author of this paper, I got funding from our trust, which enabled us to buy in expertise from Sue Roulstone, who’s here with me today, and from Paul White, also a co-author, who contributed methodological advice and statistical expertise. 

And so, with lots of interest and excellent support from my colleagues, my service got going with the study. 

HOST:                         0:04:26 A challenge, which I’m sure everyone in the NHS will be familiar with, pressure on resources created an opportunity for a clinical cohort study, in which we could track what happened with service users on the waiting list who didn’t receive any interventions in that period. 

I then asked Sue to introduce herself and talk about her involvement in this research. 

SUE:                            0:04:47 Hello, I’m Sue Roulstone. I’m an emeritus professor at the University of the West of England in Bristol, and a member of the Bristol Speech and Language Therapy Research Unit. I started off my career as a clinical speech and language therapist working mainly with preschool children and primary-aged children. I spent a while as a manager of children’s speech and language therapy services and then became a full-time researcher. 

My research interests have covered the perspectives of parents and children, evaluation of therapy services, and therapists’ clinical decision making. So, I was really pleased when Elizabeth invited me to take part in this study. It’s an area of my interest, and we were looking at, really, children who had been selected for intervention and who were on a treatment waiting list. It was a prime opportunity to have a quick look at this decision making process as well. 

HOST:                         0:05:43 I think we have touched on this, but let’s delve into the issue. What clinical questions were you trying to answer with this research?

ELIZABETH:               0:05:52 My colleagues and I had lots of big questions. So many. Fundamentally we wanted to know: do we pick the right children who really need therapy? Are the children we see that we don’t really need to be seeing or don’t need our input? How far are children’s outcomes predictable from how they presented at initial assessment? We had loads of questions, and it was quite tricky to pin things down. But for this study, we ended up focusing on three areas which I’ll just outline briefly. 

Firstly, we did a description, we described how the 525 children we selected for the study presented at initial assessment. This was essential in part so that other services could think about how generalisable our findings were to their own caseloads. Also important to give us a baseline for evaluating how the children got on, and also, importantly, to examine which children were actually accessing our service. 

The second area we looked at [recording scrambled 0:06:54] obviously what had happened to the children who came back for reassessment, having waited an average of 12 months. Most, in fact, did wait about 12 months, although our range was 6-18 months. 

We looked at the changes in the severity of their impairment, who’d improved, who hadn’t change, who had deteriorated, and how far the nature of their communication difficulties had changed over time. And by nature, I mean had their communication diagnoses changed markedly? For instance, had a child with a primarily receptive difficulty changed into a child with a primarily speech difficulty? 

And our third area of investigation, we examined the factors associated with the children’s outcomes. Outcomes such as who still needed therapy, who returned for reassessment, who didn’t, and so on. 

Just want to tell you the factors that we used. There were a number of them, and we used them in lots of different analyses. We looked at the child’s age, their sex, their postcode, the number of languages used at home, the length of time the child had waited, the diagnostic categories – we’d assigned the children two at baseline and follow up also – and also their TOM impairment ratings. 

SUE:                            0:08:18 It was just a prime opportunity with this amazing data to follow up children who hadn’t had the intervention. That was the key thing for me. But the clinical decision making in terms of do we pick the right children, are we actually picking the right children was one that I was particularly interested in. 

HOST:                         0:08:38 What does the literature say with regard to cohorts who receive interventions and those who don’t? 

SUE:                            0:08:45 In terms of answering that natural history data, the two main types of study are cohort studies, large scale epidemiological studies that follow children through over time, or randomised controlled trials. In terms of those first type of studies, the data from those cohort studies has consistently shown there’s a high proportion of children, particularly once they reach school age children, these children who present with speech and language impairments at that time continue to show a range of communication difficulties into adolescence and beyond. We know that from those kinds of studies. 

In addition, those studies show that many of these children go on to have a range of negative educational and social sequelae, like being more isolated, having more mental health problems than their peers. So, we know that from those studies. 

But the methods of recruitment and identification differ across those kinds of studies, but commonly they use a standardised assessment. They have a cut-off rating, and then it’s children who drop below that.

Now, the difficulty with that in relating that to the clinical context is that we don’t always use those kinds of assessments to give children access to our services. One wonders always, are these the same children that would come through our clinics? So, having a more clinically-based study is helpful. 

And the other kind of study that I mentioned is the randomised controlled trial, and that’s one of the studies where you contrast, for example, a new intervention or an existing intervention with no care. Now, that clearly has ethical implications, so they’re not very common. The more common thing is that a new intervention is compared with a standard care. 

But there are a couple of these randomised controlled trials. One I was party to in the late 1990s, we published it in 2000. And then the Broomfield and Dodd study, which was about 2010; they were looking at service level intervention, so they didn’t look at a particular intervention, they looked to see whether, if you offer a service, is that a better outcome for children than not offering them. Typically, you have a [inaudible 0:11:06] where the children are being monitored, but they’re not offered anything direct. 

Those two big studies – they were in quite large numbers for randomised controlled trials in speech and language therapy – both came to the conclusion that the children who were in the waiting arm, a significant proportion of those children still needed help at the end. And there were about 70%, roughly, in both studies, so that was a clue that, actually, there is something going on here. Once children come into a service, they are the [right 0:11:38] children, and they don’t get better on their own. So, it gave us faith that our study was valuable and needed expanding. We needed to know more about this. 

So, there’s some evidence about natural history outcomes, and some indications about the kinds of factors associated with those outcomes, such as poorer outcomes for children with receptive impairments. But in terms of that applicability to a waiting list in treatment, it’s quite difficult to interpret sometimes. That gave us, as I say, reason to get on with our study. 

HOST:                         0:12:13 Can we do a just a quick recap on the research design? 

SUE:                            0:12:16 The design is a simple one, with baseline data and follow up assessment after a period of waiting. The children waited between 6 and 18 months. That has already been set up, as Elizabeth had said. The clinical team that thought very carefully about their way of collecting data and documenting their decisions, so we had a large cohort of children – over 500 – who’d all been assessed using the same protocol by different therapists, but therapists who’d been shown to have interrater agreement about who they were selecting. And we did exclude those children with complex needs and those with voice disorder. 

But this was a significant cohort. Numbers are so important in this kind of study. You need enough children to have assurance that any findings you have are not just some random, lucky outcome, but have any statistical differences you find have some power to them. 

Different assessments had been used with the children because of their different presentations, so you would do something different for a child with a speech disorder compared to one who has a receptive language impairment. There had been a range of assessment used. But in addition, they had the impairment scale of the therapy outcome measures, as Elizabeth has said. 

And for those people who are not familiar with this, the therapy outcome measures was developed by Enderby and John way back – a long time ago, it’s been around for a long time. It’s been shown to have interrater reliability and some good predictive validity as well. The full scales cover the child’s impairment, activity, participation, and wellbeing and are rated on an 11-point scale, with descriptors that go from no communication difficulties at all to severe and profound difficulties. 

The other scale that we had in the data that we collected was a 7-point rating scale, that, rather than just giving you a broad overview, which is what the TOMS gives you, but it honed in on particular aspects of the child’s speech and language disorder, so on their speech, on their receptive language, their expressive language, and on their fluency. So, we had a secondary scale as well. 

HOST:                         0:14:38 We actually interviewed Pam Enderby in August of 2020, and we will include a link to that episode in the show notes if you’re interested. 

So, we come to the big question: what did the research find? 

ELIZABETH:               0:14:54 I’ll return to our three areas of investigation and give you a bit of information about what we found for each area. 

Firstly, I’ll give you a quick overview of how the children presented at initial assessment. They were aged between 20 months and 16.5, with their average age being 54 months. Most children were preschoolers, but 13% were over 6 years. A quarter of the children were multilingual, and there was the usual ratio of boys to girls of 75% to 25%. Children came from across the social range, but levels of deprivation were considerably higher than for much of the UK, particularly in some areas of the city. 

So, as a group, the children showed a high overall level of communication impairment, with 55% having profound or severe TOM impairment ratings. Most of the children had at least two areas of communication difficulty – it was unusual to find a child with just one area, usually maybe fluency. And in terms of diagnostic groupings, nearly half of the children were assigned to our receptive language category, which suggests a pretty pervasive level of impairment, compared with the other categories. 

Just to tell you a bit about the second area of investigation, child outcomes, and I am going to focus on just one key finding. There’s lots more finding in the paper, but I don’t want to bamboozle everyone with lots of figures and lots of detail. 

So, 58% of the children returned for reassessment, and the majority of this group – 83%, in fact – were still eligible for the service after an average wait of 12 months. And this is important: the average improvement in their TOM impairment rating was just over half a rating point. In fact, point five eight.

I want to concentrate on this finding because it contributes something new that is clinically important. Pam Enderby has stated that a nought point five rating improvement in at least one TOM parameter indicates that the delivery of a care package has been beneficial. To give a bit more background to this, this nought point five metric derives from randomised controlled trials, which related TOM change scores to other measures of patient outcome. And these trials focused on adult patients where spontaneous change was not expected. 

I contacted Pam to discuss this and she commented that it is important to confirm whether this nought point five metric is appropriate for other client groups. And as I’ve just said, a significant proportion of our study children made this level of change spontaneously without the delivery of a care package. 

So, in light of this, we would urge therapists to use a more stringent level to evaluate the outcome of their interventions. For example, perhaps looking for an average change of one instead of half a TOM rating point as an indicator that observed change is more likely to be related to their intervention, rather than to spontaneous improvement. 

To give a few more figures that may be helpful for services doing an evaluation, 54% of our children showed an improved TOM impairment rating of at least nought point five percent. Twenty five percent of the children showed no change in their TOM impairment rating, and 21% of the children deteriorated, in that they achieved a TOM impairment rating that was at least nought point five more severe than when they’d been initially assessed. 

We analysed our outcomes in a number of ways, and we’ll discuss all of that in the paper [inaudible 0:18:58] the groups of children who are more likely and less likely to improve, and I won’t go into that there. I just want to point out that there was a lot of overlap in our findings. For instance, whilst children with milder difficulties were more likely to show spontaneous improvement, some children with profound levels of impairment at baseline improved a lot, and some children with mild impairments deteriorated a lot. 

Finally, a word about how far children changed in terms of the nature of their impairment. That is whether they changed the diagnostic category that we assigned them to. Twenty percent actually did this; 80% didn’t, but at 20% a significant minority did. And just looking at those children who did change diagnostic category, I suppose, as you will probably predict, those with an improved TOM impairment rating tended to move from a more pervasive to a less pervasive diagnostic category, and those with a more severe TOM impairment rating tended to move in the opposite direction. 

I do want to thirdly say something about the children who didn’t return for reassessment, because they were a very important and a large group. It was a study carried out in a service context, so we didn’t follow these children up routinely because that’s not what we do in a service context. So, 42% of the original cohort of 525 children didn’t return for reassessment. The characteristics of this group had mostly been very similar at baseline to the children who had returned for reassessment. 

We had no reason to assume that the communication difficulties of the children who didn’t return just resolved spontaneously. And in fact, an audit carried out shortly after the study had been completed found that 9% of the non-returner group had, in fact, been returned, re-referred, and already re-admitted to the caseload. 

When we looked at factors, the only factor that did differentiate those who didn’t return from those that did was that a larger proportion of the non-returner group lived in the particularly deprived areas of the city. 

HOST:                         0:21:12 Lots of really interesting findings there. This spontaneous nought point five improvement in the TOM scale is really interesting. But if I understand you correctly, Pam Enderby is saying that that’s perhaps less surprising in a paediatric cohort. Could you just tease that out for us a bit? 

ELIZABETH:               0:21:30 I think the interesting thing is here that we as, paediatric therapists, our client group is developing and so you’ve got developments on your side, unlike maybe adult clients where maybe that’s not the case to the same extent. 

I think the thing is that we just didn’t know how much spontaneous improvement children would make because how would we know? We don’t tend to leave children for a year and then we assess them to see what’s happened. And so, this is this is really an important part of what this study can contribute. My point that I really want to get across is that I’m hoping services will use this information when thinking about whether their interventions have been successful. If they only get a point five change overall with their caseload, I suppose we’re saying, well, actually, you probably would have got that anyway, or we did at least in this study. So, it’s just giving a bit of a benchmark for services and they’ll decide for themselves how close their caseload is to the one we’ve described, so that they can evaluate the effectiveness of the input that they have provided. 

SUE:                           0:22:45 Just a very quick point about that point five, as Elizabeth has said, but just to stress, that’s the average across the caseload, and there’s a lot of variation between children, individuals. 

So, when you’re interpreting that, as Elizabeth has said, it’s important to think about if I’m interpreting the progress made by my caseload as a whole, that the point five is not necessarily a critical point for individual children. And we weren’t able to establish in this study, which is why it’s not in the results anywhere, a particular, a critical point at which we can say if they’ve made one point improvement on TOMS then they’ll be okay. That will be the point at which we can say they don’t need intervention anymore. That point for the individual is hugely variable, and we’re not able to say what that is. But thing is, when you’re evaluating your caseload as a whole that you can say if it’s only point five for your caseload as a whole, then they’re likely to have made that progress anyway. 

HOST:                         0:23:51 Aside from the question of the spontaneous improvement, what else do you think we should be aware of? 

ELIZABETH:               0:24:00 I think the finding that it is the children from the more disadvantaged areas that don’t come back at follow up is also something that we know but we don’t necessarily collect evidence for. But the baseline data, the percentage of children that we lost was heavily skewed towards those children from the more general disadvantaged clinic areas, the areas of the city. And so, that whole idea of having a treatment waiting list significantly impacts on those children from disadvantaged areas. It adds another layer of disadvantaged if you require that of children in your management of cases. So, I think that was that was another thing that we felt was really important. 

I think there were also findings to do with trying to sort out what impact that poverty perhaps has. For example, you have to understand the demographic of your caseload. For example, the bilingual children in our cohort had significantly poorer receptive language scores relative to the monolingual children. But we had more of the bilingual children living in areas of poverty. Trying to tease that out of our data wasn’t possible. We have to be cautious about how we interpret the fact that bilingual children seem to have poorer receptive language when we know that poverty is impacting on that as well. 

HOST:                         0:25:26 Do you have any recommendations for practice or further research in this area? 

ELIZABETH:               0:25:31 I’ll give you a couple. This gives me a chance to air one of my frustrations when doing this, looking at the literature. A recommendation would be for the SLT profession to agree a standardised method of assigning children to a diagnostic category, and this would make it so much easier to compare results across studies which have a natural history element to them. 

When I was searching the literature to look at what we wanted to use for our diagnostic categorisation, I came across various systems and I eventually decided on an approach that was influenced by Dorothy Bishop’s hierarchy of vulnerability. 

But my main point is that I was frustrated whilst choosing this at the fact that we were developing yet another slightly different system of diagnostic categorisation, which would inevitably make the findings from this study not directly comparable with previous studies that had a natural history element. 

The second thing would be to encourage services to standardise their practice in terms of assessment protocols and methods of recording, so that data can be extracted relatively easily when evaluating some aspect of service delivery. And I say this, because I’m so conscious that the data used for this study was recorded on one sheet, one case summary sheet, which – we had paper notes at the time – was always placed at the beginning of the case notes and so we were pretty easily able to extract that information from hundreds of case notes. 

The study, in spite of all the work we’ve done just would not have been able to take place. If we hadn’t had that it would have been too big a task to trawl through hundreds of sets of notes. 

SUE:                            0:27:19 I think Elizabeth's call for a standardisation of assessment protocols and methods of recording supports the RCS Lt strategy to develop the national outcomes database. That's the route database having a national outcome measure that everybody agrees to use and contribute information to that not only enable services to benchmark their outcomes, but it facilitates the gathering of data in research studies too. In fact, Rcslt has been supporting other studies.

To achieve consensus about the particular outcomes and measures that are appropriate for different client groups. So for example, the study led by Professor Yvonne Ren, it's called the MISTLETOE SSD study has been working to establish core outcomes set and a minimum data set for children with speech sound disorders that process the process that that studies established to identify consensus may well be applicable to other diagnostic groups too.

HOST:                         0:28:21 And are there any limitations that you think we should be aware of? 

SUE:                            0:28:25 It is a clinical cohort, so the assessments had been decided upon by the clinical team, so we weren’t in a position to impose additional assessments and to plan those after the event. And we had only the one outcome measure, the impairment scale of TOM. Clearly, it would have been much more useful if we’d have had the other scales as well and been able to include the participation and activity assessments. I think that’s the main one. 

The cohort was based in this high area of disadvantage, so although that gives us really interesting information, the proportion of multilingual families in there is not necessarily generalisable to all UK context, so people need to take that into account when they interpret it for their own clinical context. 

HOST:                         0:29:11 Is there a final take home message for SLTs? 

ELIZABETH:               0:29:14 It’s a positive one. In this study, only 17% of the children who came back didn’t need further intervention. So, after a considerable time waiting, 83% did still need the services of an SLT, which is a similar finding to the Bristol and to the Middlesbrough studies that I mentioned earlier. 

This suggests that in spite of all the variation and developmental noise and difficulty with prediction, therapists do generally pick out the right children, those who have more persistent rather than transitory difficulties. 

HOST:                         0:29:56 We end on a heartening message that caseloads, we think, are broadly right. 

A reminder to listeners about the RCSLT online outcome tool. It’s a tool that allows you to upload anonymised outcome data, run reports, and gain insights into your service, and it also allows you to compare your service to the national data, so it’s a great research tool and really useful for service evaluation, etc. 

And given the current pressure on the NHS, the one silver lining of waiting list is that it does provide interesting research opportunities to test the efficacy of interventions and the natural progression of disorders and so forth. 

A very big thank you to Elizabeth and Sue for their time. As always, please see the show notes for links, rate, review, and share the podcast to support the work of SLTs in the UK and beyond. 

Until next time, keep well. 

MUSIC PLAYS: 0:30:50

END OF TRANSCRIPT: 0:31:02