Exploring AI's Impact on Autism Care: Tools, Challenges, and Surprising Results

As a parent navigating the challenges of supporting an adult with severe autism, I took a closer look at AI tools—ChatGPT, Claude, Perplexity Pro, and Gemini Deep Research—to see how they could help parents like us. ChatGPT and Claude gave general advice, but Claude caught my attention by sharing relatable fictional case studies. Perplexity Pro provided references, but it felt like you had to dig for the gold. Then there’s Gemini—it delivered tons of info, including research plans and real-world examples, though not always specific to severe autism. While none fully captured the unique challenges we face, Gemini stood out for its sheer amount of helpful content, making it my top pick for research, even with its quirks.

Transcript

Mike Carr (00:07):

This week we're going to talk about something that Sunder Phai, who happens to be the CEO of Alphabet, a trillion dollar market cap company said about ai. Here's the quote. Artificial intelligence will have a more profound impact on humanity than fire, electricity, and the internet. So AI promises to change the world. What about the world of profound autism? Is it going to improve the quality of life of your adult profoundly, severely autistic child of mine who's 35 years old? That's what we're going to answer today. And I've done a lot of research on ai. We're going to look at four of the top AI tools on the market today, and we're talking about ai, not so much from, oh yeah, I can use AI to power a robot, that my son or daughter's going to be excited, or I can use AI for education, or I can maybe even use AI instead of a one-on-one colleague.

(01:07):

That's not what we're going today. What we're going with today is something you can use right now immediately, and it doesn't take a lot of effort. Just a couple prompts. It's all about research. It's all about finding resources and tools and programs and techniques that are specific to your need. So that was the challenge that I wanted to put four of the largest leading LLMs, put them up against the test and see how they did. So the first one I tried was chat, GPT, the granddaddy of all of 'em, right? Been around for everyone. Everyone's wrote chat, GPT. The second one's, Claude Claude's, my personal favorite for writing and editing, not as well known as chat GPT, but still a pretty well established LLM. The third one was Perplexity Pro, and that's also one that has a reputation for doing a really good job at research, giving you lots of good citations.

(01:52):

And the fourth one is actually the newest and the one that I was most excited about. I'd heard some interesting things about it. It's actually from Google, and it's called Gemini Advanced 1.5 Pro Deep Research, sort of a mouthful, but it's basically Gemini Deep research. I started with chat GPT. Here's the prompt. I gave them this exact prompt. I am writing a book to help parents of a severely autistic adult child transition their child to adulthood. Can you research and find the best papers, the best books, the best podcast and the beds programs in the world, and provide a complete plan to help parents successfully make this transition? So that was the prompt. If you key that in and chat GPT, you're going to probably get something very similar to what I got, which was about two pages, about two and a third pages, and it was just too generic.

(02:47):

It was just general stuff about autism, very little about severely autistic or profoundly autistic. It was the kind of stuff you could find really going to any of the national associations, the Autistic Society of America. They have transition planned outlines. It was about that level of detail, basically a lot about, yeah, you can do this or that, but not hardly anything about how to do it. And so I was a little disappointed. So I felt that chat, GPT needed to be reprimanded. So I said, chat, GPT, bad boy, bad chat, GPT. And then I gave it a different prompt. I said, the research you compiled doesn't appear to focus specifically on those with severe, profound or level three autism like I asked you about. I didn't say that last part. This population is much more difficult to find information and help than the general autistic population.

(03:38):

They typically have an IQ of under 50 or nonverbal and often require one-on-one care. Could you redo your research addressing specifically the much more challenging problems parents of an adult child with severe autism face? And so it came back with about another two pages. It was a little bit better, but it wasn't that much better in terms of really focusing on severe autism. But one of the things I did, which I thought was really pretty cool, is it knew I was in Austin, Texas. I didn't tell it this. It definitely knew that from my IP address or something where I was located, and it came back this time with programs and resources in Austin or around the Austin area, and that was pretty cool. Now, I'd heard of most of these, but there were a couple I had not heard about, and the fact that I came up with those and suggested them even though they weren't specifically for severe autism was pretty cool.

(04:28):

And so I was pretty excited about that. Then I went on over to Claude, and I gave Claude the same exact prompt that I gave chat, GPT, and I was disappointed. It started with what I thought was a bit of a disclaimer. So it started right out of the shoot with I don't have access to current databases or search capabilities, so I can't guarantee the completeness or accuracy of specific citations. Then it went on even with more of a cop out. Instead, I'll focus on providing a structured framework based on well-established principles and areas that parents should focus on. And so it gave me about two pages, about two and a half pages, but it was a very high level framework, sort of like what chat GPT started out with. So I gave it the second prompt. I reprimanded it, and I said, no, I need something that's a little bit more specific.

(05:18):

I understand that you don't want to give me too much because you aren't current and you don't have all the most recent papers, but can you come back with something a little bit better? And it didn't do much better, but it did do one thing that was sort of cool. It gave me three case studies, three people that were severely autistic and explained their symptoms and then explained the programs, the steps that those parents took. I thought that's interesting. So I looked at those. It didn't give me those initially. It said this was one more question you could answer. And the first thing I start with is, well, these are fake. But it told me that it said, these aren't real people. These are composites that I made from lots real people that I researched it, blah, blah, blah. Okay. And then it gave me the success strategies.

(06:02):

And this was sort of interesting for Miguel, I think Miguel was 17, this fictitious person started the transition plan at age 16, gradually introduced parents, gradually introduced respite care workers who later became permanent staff, which is what Kay and I did when our son was in that age range, created detailed care protocols for all medical and behavioral needs. We did that too. Very helpful. Used video modeling to familiarize McGill with new environments. Great idea that also worked well for us. Spent 18 months visiting potential group home before selection. We'd spent about two weeks, three weeks traveling the country, and we didn't find any group homes that were any good, but 18 months maybe, I don't know. And then implemented a six month transition period with overnight stays increasing gradually. So these three case studies that you had to sort of query it to give you we're sort of interesting, even though they were fictitious.

(06:52):

So I would say Claude wasn't as good as chat. GPT surprising in this one area that chat, GPT just didn't even come up with Complexity Pro, which I'd used in the past for Research Promise. This comprehensive plan, I gave it the same exact query. It was going to give me this comprehensive plan, and this comprehensive plan cited one research paper, one podcast, and two programs. That was it. It's like, give me a break. So I asked it again to do a little better job, and it came back with some stuff that was a little better. I went out and checked some of the sources that it gave me, and inside those sources, there were references to things that were of value, but required me to do the digging. So Perplexity Pro cited some references that I was able to go out and check further and find things inside of them that were relevant and were something that I was not aware of or I found helpful, but it didn't highlight that, right?

(07:49):

It is like I had to go do the digging. So it was helpful in finding some of the resources and references to check and citations, but it didn't fare out the stuff that I wanted. So none of these three had really done what I was hoping for. So I went to Gemini Deep Research. I gave it the same prompt, and it gave me a research plan. I thought, this is sort of interesting. It gave me an eight point research plan, said, this is how I'm going to go answer your questions. I thought, this is sort of interesting, right? Told me step-by-step how I was going to do this and had all these websites I was going to research. This is pretty cool. So I said, go do it. And they said, well, do you want to change anything for us? I said, no, I don't want to change anything. I want to see what you can do. This is your research plan, not mine. You go do it. It took it a couple minutes and went off and thought and do whatever, did, and computer started smoking. The computer didn't start smoking, but could have started smoking. They came back with seven and a half pages, oh my God, three times, four times as much as anybody else could come up with and all kinds of references and books and programs. This is great. Started looking at it. Generic autism stuff.

(08:51):

So I gave it the same prompt. I reprimanded it. The research you compiled doesn't appear to focus specifically on those with severe, profound or level three autism. This population is much more difficult to find information to help for than the general autistic population. Gemini, they typically have an IQ of under 50 or nonverbal and often require one-on-one care. Could you redo your research addressing specifically the much more challenging problems parents and adult child with severe autism face? Now, I was getting a little red in the face at this point, but it did not see that. And so it went off and thought for a few more minutes. Actually, before I did that, it gave me its eight point thing in each of the things it was going to do. It talked about how it's going to do this for severe autism or profound autism. But the last three, it left that out.

(09:38):

I edited its research plan at this point, and I stuck in there severe autism, severe autism, profound autism. I made sure every one of the eight things it was going to do was specific to the population. And then I turned the thing on and it went out and fought for a while, and it came back with 12 and a half pages this time, more than the seven and a half. Oh my God. So I was pretty darn excited. I looked at all this stuff. Some of the references were generic, some of the references I had not thought of, and they were specific to severe autism. However, some of them were just totally bogus. I wouldn't say they didn't exist. They were just of marginal value and weren't that specific to anything I was interested in. And then it came up with six case studies. So remember Claude came up with three of fictitious people.

(10:28):

Well, Gemini came up with six, and these were real people. It actually gave you the names of the real people. And the first one to start with was Susan Boyle. I thought, I know who Susan Boyle is. I watched her on America's Got Talent or something. And she's a Scottish singer who's got this incredible voice, but she has Asperger's. And I'm thinking, well, somebody with Asperger's isn't really on the low end of the spectrum. At least they're not considered severely autistic. I mean, they don't have an IQ of less than 50. They certainly can talk. She can sing and talk. So even though she was a great example of someone who'd been very successful with autism, she was not severely autistic. And the next person was Ann Risi. And she got a master's in college in social studies and was teaching it. And I'm thinking, well, she's probably not severely autistic either.

(11:17):

And then the other four I checked out and sort of the same way, these were all real people and it cited who they were and a little bit about their accomplishments, but none of them were severely autistic. So in conclusion, I would say that collectively, because I've been researching this stuff for a few weeks for the book I'm working on, they all did find things that I had not found, and they certainly found things in much less time. I mean, I'd spent a couple of weeks and they spent a few minutes. So from a research standpoint, definitely a value. They each found something of value that the other three had not found. So it's sort of like you can't just rely upon one. Since all of them sort of came at it from a little different angle and came up with some things that were interesting.

(11:59):

None of them though, even Gemini deep research, really understood the subtleties and the nuances about how profound autism and severe autism is a very different animal than the autism that three-fourths of the autistic folks out there have, right? The folks that can talk, the folks that still have a lot of autistic behaviors and challenges, but they can function much more independently. Whereas profoundly and not severely autistic folks, our son requires one-on-one care from the time he wakes up in the morning, the time he goes to bed. We know other people that require 24 hour care because they have other medical conditions. So it's just different ballgame. None of the LLMs really understood that. But I would say of you have to do one, you only have time to pick one Gemini Deep Research hands above the other three. I mean, just the sheer volume of stuff.

(12:45):

It produced 12 and a half pages, and this is really good stuff. I mean, it gave you web links, it gave you the names of the organization, it gave you the programs, it cited the research papers, it suggested all this stuff that you then could go out and pick the ones you felt were the most appropriate. Now, some of them were state specific, and it told you that some of them were not. And then you can go into and do some additional research yourself. So it wasn't like a final answer, but it was pretty interesting and certainly a pleasant surprise, especially after I tried the other three, and I was a little bit disappointed. So I'm sure by the time a few more months go by, AI is probably going to get better. It's improving at an unbelievably quick pace. So I certainly wouldn't give up on ai, and I did find this exercise of value and helpful, but you have to check everything.

(13:30):

You have to go out and do a little bit additional research. I would say it saved me a lot of time, but I still had to sort of ferret out the stuff that wasn't relevant or that I already knew, and then focus on the other things. So in terms of grades, I would've given perplexity, pro and claw, probably D's or D minuses. I'd probably give chat GPT, maybe a C plus or a B minus. But I probably would give Gemini like a maybe a b plus, a minus. It wasn't an A or an A plus. It was still getting things wrong. It was still citing irrelevant sources, but it was surprisingly good. That's it for this week. Talk to you again soon. See you.