Episode 7: The Role of Assessments in DPT Programs

ACAPT’s newest episode of FlexCast examines how assessment practices are shaping program quality, student outcomes, and strategic decision-making in Doctor of Physical Therapy (DPT) education.

Hosted by ACAPT Board Director Sara North, PT, DPT, PhD, MEd, FNAP, University of Minnesota, the episode features Katie Myers, PT, DPT, PhD, Duke University; Rebecca Moore, PT, DPT, cert MDT, OCS, University of Memphis; and Ashley Parish, PT, DPT, PhD, CRT, University of Alabama at Birmingham, leaders working directly in program assessment and outcomes management. Together, they discuss the growing importance of dedicated assessment leadership within academic programs and the role assessment plays in supporting accreditation, curriculum alignment, and continuous improvement.

Transcript

SPEAKER_01 0:00

Welcome back to FlexCAT, where ACAPT explores the ideas and innovations shaping the future of academic physical therapy. I'm Emily Weber, Director of Marketing and Communications at ACAPT. Today's episode focuses on something every program does, but not every program is structured to do well. Assessment. From accreditation requirements to student outcomes. Assessment is at the core of how programs measure success, make decisions, and continuously improve. We're joined by leaders working at the forefront of assessment in DPT education to explore what this role looks like in practice, why it's essential, and how programs can build more effective data-informed systems. Welcome, Drs. Katie Myers, Rebecca Moore, and Ashley Parrish, along with our host, ACAPT Board Director, Dr. Sarah North. Welcome to FlexCast.

SPEAKER_03 1:00

Thank you so much, Emily. This is going to be the best episode yet. No offense to the previous episodes. We are very privileged to have three fantastic representatives from our profession to speak about assessments today. An area that is confusing, murky, and doesn't have a very clear path. So to get started, I would love for each of you to please share your title, your role, and maybe explain a little bit about what that entails and how you are coming today to share your expertise on assessment. Katie, let's start with you.

SPEAKER_02 1:32

I am the director of curriculum for Duke's DPT program. I also serve as a co-director of clinical education in the program as well. I've been in the director of curriculum position for a little over a year now and still getting my feet wet in terms of what the role entails, as it's a big role. And something that's different about our program, which I think we'll explore as we get into the podcast, is we don't have a specific director of assessment. So I really command to this from the conversation of what is assessment from a curricular standpoint look like when you don't have a central director of assessment role. So I'm really in my role ensuring that the curriculum is aligned with CAPTE standards, you know, at first and foremost, and is coherent and that there's integration across didactic and clinical professional domains, and supporting faculty in their course design and assessment strategies for their courses, and really using assessment, curricular assessment data and program assessment data to inform our curricular decisions. So really interwoven assessment roles within curriculum that really tie closely to programmatic assessment and outcomes.

SPEAKER_00 2:55

Hi everyone, I'm Ashley Parrish and I'm the director of assessment at UAB. And this is a new role for our department that was recently created within the last year. And I officially stepped into the role early in 2026. So in my role, I have a bit of a different role. As you'll see, we all kind of do, but I oversee assessment for our four programs. So we have our DPT program, our PhD program, a bachelor's program, and a certificate program. So we have a committee with all of our program directors that sit on that committee. And together we are the assessment committee. So we do everything from CAPI accreditations to SACS accreditations, but my largest involvement is assessment of the DPT program. Fantastic. And Rebecca.

SPEAKER_04 3:43

Yeah, so I'm an assistant professor and the assessment chair at the University of Memphis for our developing DPT program. I say developing, but fingers crossed, we should be hearing any day now and ready to seek our first class in August. But so in addition to my teaching role, I'm going to be serving as the assessment chair. And I will be seeing overseeing both a programmatic and program assessments. So right now, this includes me building a system to collect meaningful data, analyze it, and report it to faculty and leadership so that they can make actionable decisions. So we're very blessed. Our program director recognized early that assessment has to be intentionally designed. And that's where and how I get to contribute to our program.

SPEAKER_03 4:32

Excellent. We clearly have a really diverse set of program representation, and that is where the richest conversation comes in. That is wonderful. So one of the things that I think is really helpful to hash out is what does it look like to be in these roles? Not all programs have a clear role, as we've already identified today. Sometimes it interweaves with other identities within the program.

SPEAKER_02 4:55

So my role day-to-day, and you know what my role looks like on a day-to-day basis is obviously very curriculum-focused. And I think what we'll hear from Ashley and Rebecca is where assessment goes beyond curriculum, obviously, is for the program. But of course, they're very linked. And so I do a lot of interfacing with our program director to talk about our needs to support our curriculum based on our data and our assessment data. So I'm looking at course evaluations, I'm looking at student performance trends, I'm looking at clinical education feedback, especially as co-director of clinical education, and also meeting with faculty, particularly through our curriculum committee, which I'm the chair of, to really intentionally evaluate the data that's coming in so that we can look for needs, gaps, opportunities, challenges that we need to address or that we need to make sure we continue to do because we're doing well. And overall, just really supporting just continuous improvement. I think curricular curriculum are not is like never static, it feels like.

SPEAKER_04 7:30

Yeah, thanks, Katie. Um, so right now we're building the plane, right? Or new. So I'm building the infrastructure of what our assessment looks like. So clarifying what we need to evaluate and those timelines. Like we don't want to layer it on afterwards. We want to begin to integrate this early on. And that's the benefit of starting this now as we're developing program. Ashley, I know that you have an established program, but you're also kind of this role is new. So what does your day-to-day look like?

SPEAKER_00 8:00

That's a great question. I hate the phrase it depends, but it that is really true. It's very situational day to day. So I do have an allotted workload towards assessment, which is probably something we'll get into a little bit later in the podcast. How I keep myself organized and running is a calendar. So I have a massive calendar of our assessments, when they go out, when they end, when I do the um the analysis of the data, when we share that with faculty. So that's how I keep myself structured in terms of the day-to-day. Again, I'm busiest with our DPT program because we do have our CAPE accreditation, where some of our other programs don't have that formalized accreditation process. So it is a lot of talking to the program director and communicating things like Katie mentioned in the day-to-day of this role. But there are certain times where I am busier, like around our graduation and when our new cohort comes in, where we're collecting a lot more data than other times.

SPEAKER_02 9:00

I think that's a great point, Ashley. I I um just before coming on to the podcast, I'm actually editing a curriculum survey for our graduates. And I think that ebb and flow is really important to acknowledge. And probably Rebecca, as you're building the ship, as you said, thinking about that timeline of the when assessment happens, so that it's really obviously intentional, like intentionally um scheduled around benchmarks and um progression in the program. But that's why I mean there is such an ebb and flow to what our quote unquote day-to-day looks like. Um, sometimes it's very busy, especially as you said, Ashley, around graduation, um, or we have sort of phases in our curriculum. So as phases end and we're looking at um academic progression and benchmarks, that's another area that's big for us.

SPEAKER_03 9:52

Nice. Now, one of the things that we often wonder in our profession, what does effective assessment mean? And Katie, you pointed out earlier, assessment, evaluation, these are actually different terms. And there is curricular assessment, there are faculty assessments, student assessments, then there's the overall program evaluation that often incorporates multiple aspects of assessment. And so I'm curious if anybody has a thought on how do you define effective assessment in a DPT program?

SPEAKER_00 10:23

I have a very short answer for this, and it's just collecting the right data, analyzing it, disseminating it, and then using it as an opportunity to change for improvement in the program.

SPEAKER_04 10:35

Similar. Thanks, Ashley. That like almost verbatim, kind of what I had thought of with in my mind of effective assessment is collecting that meaningful data to provide your leadership with the validity evidence to make informed decisions about the program. So student progressions, curriculum, program outcomes, like all of that supports student growth. Um and of course accountability standards like CAPD.

SPEAKER_02 11:03

Yeah, I think about it and Sarah, I love that you pointed out that there's assessment and evaluation because I even think about what we teach our students, right, with exam and evaluation and what the differences there, those different processes. So assessment to me is like we're collecting the data, we're doing the examination, we're assessing in whatever the areas that we're assessing. And that's all informing how we are looking at the overall decisions and next steps in our evaluation of the program. And hopefully it's all intentional. So, like my little answer to how you define effective assessment would be it's intentional and it's integrated and it leads to meaningful change. But that means you have to collect the right data and then you have to have the right or or you have to have a clear path as where you're going when you're making those evaluation kind of decisions of are you are you meeting the mark or not? But assessment to me is like collecting whatever that data is, um, whatever that feedback is, whatever, whatever you are looking for, that's going to help you really inform those decisions.

SPEAKER_03 12:14

I'm hearing a lot of synergy between all three of you on that understanding, which is great. And I'm also hearing an undertone of that backwards design. If you understand where you are going and you truly do work backwards to identify and perhaps fill gaps in areas that you want to collect assessment information, I'm gonna back that up, in areas that you want to assess uh components of your program, but maybe you have gaps there. So that quality improvement. So, what happens for programs that are out there that don't have a dedicated role focused on assessment? What are some of the challenges and opportunities that might arise for those programs?

SPEAKER_02 12:52

Well, we are one of those programs. Um we haven't always been one of those programs. We um have had a director of assessment position in the past. And in the recent years, we've had more of a um what I would call a distributed model of assessment where we have different leadership under different um areas of the program and within our division. So, director of curriculum, we have clinical education, student affairs, um, culture and leadership, um, we uh admissions, and then of course our program director. And I think the challenge is if we go back to the answer to that question of effective assessment, you want integration, you know, meaningful um data and approaches to collecting data and that you're all moving in the same direction. So the challenge is to make sure that if you have multiple people involved in assessment and data collection, that you're all aware and on board with where you're moving in that direction. Um, the opportunity there is there's lots of different perspectives and ideas, and you have the ability to not just feel siloed in your perspective of curriculum or clinical education or even just overall program outcomes. You actually can, I think, as you engage in these conversations about what your data is showing you and these different aspects of your program, that it can really help everyone think broadly and really meaningfully about where you're going as a program and what the, again, the gaps and the opportunities are to continuously improve the program, um, the outcomes of the student, your student outcomes, the type of PT that you're preparing in your program and and who is who's going to leave your program to contribute to our profession.

SPEAKER_04 14:44

Yeah, Katie, I agree. I think the other thing too is just maybe programs might miss a way to see the longitudinal patterns. Um and that's important if you're talking about integration and student growth and and progress, um especially catching things early on. Um, certainly not a challenge that can't be overcome, but it is a challenge.

SPEAKER_02 15:06

I think one thing I would just say is the other challenge I think um that can arise. And I and I I know personally, I'll just speak to my own experience of coming into the director of curriculum role. But prior to that, I was assistant director of curriculum and still in the DCE role. We went through CAPTE accreditation or re-accreditation, and um it really opened my eyes to how everything is tied together. There's so much linkages between your program outcomes, your curricular assessment uh approach, your student outcomes, all of it is linked. And I think if not everyone on faculty is aware of that and understands how we how everything moves in the same direction. Um, and really at the day-to-day level, at the course level, all the way up to the program level, then it can really result in confusion or can result in opportunities for data to be missed, you know, opportunities for meaningful assessment to not actually happen because people aren't really aware of what should be collected. So I will just say as hard as CAPT a CAPTE reaccreditation is, it was super, super helpful for myself to really understand why assessment at all these different levels is so important. Um, and being on the same page, whether you have one person in that role driving the ship or multiple people rowing the boat together, um, how meaningful it is, how important it is that everyone is on the same going in the same direction.

SPEAKER_00 16:48

Yeah, I couldn't agree more. And and when I got this role formally, I contacted Gail Jensen. And the number one piece of advice she gave me is one person cannot do all of assessment in a DPT program. And that's really true. Um, I think some of these director of assessment roles, it's important to see this not as that person doing everything with assessment, but like you mentioned, Katie, having dedicated people, like I have an admissions chair who deals with our admissions data, but as director of assessment, I oversee that and having all these parts and pieces come together collectively. And that's why it's so important to know the right data to collect, because it can get very overwhelming if you have too much and not the right things to assess. So that's really the main challenge that arises is kind of putting all the puzzle pieces together because assessment is massive and it can be even bigger depending on what you want to want to assess.

SPEAKER_02 17:44

Ashley, can I ask you a question about what you just said there in terms of like as an example, working with admissions chair? What does it look like in practicality to oversee the admissions, you know, data assessment? While that admissions chair might be leading, um, what does it look like as a director of assessment to oversee that?

SPEAKER_00 18:07

Yeah, that's a great question. So we actually have our staff involved as well. So they're very helpful to pull data from PTCAF. So we do have a data set, and I work with our admissions chair on kind of keeping that up to date and running trends. Um, and since I took over this role, we're talking about a dissemination plan where we share that data at appropriate times with the faculty, just so everyone is kind of up to date on what our admissions processes look like. Um, and we're starting to do some pieces on assessment of our admissions. So we use an asynchronous platform for admissions. So that's something that we're currently assessing right now. What are the faculty's thoughts and perspectives and the students going through that? Because we are a residential program and we're not doing in-person interviews like a lot of programs do. So that's an example of how I'm kind of overseeing that aspect of as director of assessment.

SPEAKER_03 18:59

So you have all highlighted the need for cross-collaboration in a program. There are a lot of players in the game. We have faculty, we have various committees, we have the program leadership. For those listening who are outside of PT education and those within know this very well, our DPT programs tend to run fairly lean. Our faculty play multiple roles. We are content experts, we are instructional designers, assessors, researchers, we provide service within and outside of our programs. We do quite a bit. So I'm curious how each of you see your roles supporting faculty and the program. How do you communicate your work to all of the faculty? How do you engage them in your work?

SPEAKER_02 19:40

Well, I'll speak from the curricular side of things. On a routine basis, during our faculty meetings, I will share what the curriculum committee is working on, what our initiatives are, um, what our decisions have been, what are and what's informing those decisions or recommendations. We really want to make sure that faculty feel a sense of connection and ownership to the curriculum and don't feel like they're just being told what's happening or what to do. And so the curriculum committee really tries to thoughtfully look at data, um course eval data, our survey data, or PTGQ data, all these things, and to come forward with recommendations that are then brought for discussion to faculty to ensure that there's understanding and buy-in about the why behind why what decisions are being made or changes or opportunities that we've identified. So I've really tried to be clear and around what is informing those conversations from an assessment data standpoint. Um, so that's one way that we do it. Um, just again, speaking from the one perspective of curricular chair and curricular assessment. And to your point, um, Sarah, faculty play a lot of roles and our faculty teach a lot and are part and involved in a lot of aspects of the program beyond just teaching, they are advisors, they are um, you know, contributing to a lot of efforts within our program. And so we want to be sure that whenever there are recommendations, whenever there's opportunities for change, whenever there's discussions about needs, that it doesn't appear that it's just coming out of nowhere, that there is actually data and a reason behind it.

SPEAKER_00 21:38

Yeah, and I think to Katie's point, sometimes this gets a little bit tricky because you don't want people to take things offensively. Um it is hard to disseminate some of that data from time to time, depending on the situation.

SPEAKER_04 21:54

I I think one thing, since we're so new, what we're trying to do is forget the term as Assessment shared frequently and get faculty involved from the beginning. So shared language, shared experiences from staff meetings to curriculum meetings. So it's a part of the identity of the program and not just necessarily somebody's role. And I think that's kind of how those efforts across our courses and even into Clin Ed can kind of bleed into each other. And I just think integrating it deeply and looking at it just a little bit differently, not necessarily as a role, but kind of part of what the program is and through those, you know, shared expectations is helpful.

SPEAKER_00 22:38

It's creating a culture of assessment. Yep.

SPEAKER_02 22:41

I think transparency and like visibility of the assessment processes helps with that. I was just thinking that same phrase, Ashley, of culture of assessment. Um or I also think about data-informed decisions. You know, we we talk again, you know, we teach our students, we talk a lot about evidence-based practice. And I'm I am a big proponent of data-informed decisions when it comes to any kind of programmatic change, curricular change, anything that's going to impact our student experience, our faculty experience, our program outcomes. I want to know the why behind it. I want to understand what's informing that identification of the opportunity. And I think most people are, most faculty are. We also find that our students want to know as well. Um, even though they may not benefit from the change, you know, the feedback that they provide along the way, we try to provide summaries to the students around our curricular surveys that we do at different points of time to really create visibility of how we go about making decisions in our program related to all things, you know, whether it be curriculum or outside of that.

SPEAKER_00 23:56

What about in terms of students? Do you all disseminate any data to students or plan to, Rebecca?

SPEAKER_04 24:02

Uh that we're we're still discussing that. Um, you know, that's a big, that's another great topic uh for assessments is when and what data to share. Um some things are better left internally. Um I think you know, it's important for students to understand, like Katie said, the why behind any decision making process that the student is included. Um maybe even some curricular stuff. Um certainly they're gonna see outcomes as reported on websites and things like that. But I think that is going to be a routine discussion that we have in our faculty meetings is what to share and when it's appropriate to share it.

SPEAKER_02 24:44

I think what's an interesting point there, and this may be taking us down a road that we may need to come back from, but um I I think some of the conversations we've had in our program around sharing data, you know, feedback, course eval feedback, sharing um, again, we I I've been referring to a curriculum survey. So we do these phased surveys at the end of different components of our curriculum. And it's not just about the curriculum, it's also about experiences within the program, their relationship to faculty, their, you know, all kinds of things. One of the things that we've talked about is that sometimes sharing that information with students might help students understand that their experience may be different than others or maybe similar, you know, depending on where they are, right? So often, you know, there are times when the loudest voices may not represent the majority. And I have had conversations in the past with students who have said, you know, sometimes it feels like decisions are made based on the loudest voices who are saying something needs to change when maybe the whole program, like the rest of the students or the majority, don't feel that way. And it was a that particular conversation was really important for me to hear because I thought, well, that's interesting. That means that they might have a perspective that changes are being made or decisions are being made that are not based on full data or a full, you know, a capture of really what the cohorts are experiencing, what our graduates are experiencing. Um, and so I think it's, I think there's a line. I think there's benefit to being transparent. I think it helps feed a culture of assessment. It helps promote them to complete their course evals and participate in surveys and participate in the exit group focus groups at graduation and things like that. Um, even if it doesn't benefit their particular experience, they're contributing to this culture of our program and the community of our program. So trying to make it part of this sense of belonging within our program, of being a member of the Duke DTT community, I think is an opportunity, but there is a line, right? There are things that maybe you don't share or things or how you share them is important as well.

SPEAKER_03 27:06

If we're honest, data is really only useful if it's managed well. That's come up a couple of times. If you're collecting information that you don't plan to use or you don't have a clear connection to your mission, your program's values, your program's expected outcomes. It's extra data, if you will. And so being very strategic about what you're collecting is a key part of that decision making. So we've heard from each of you a little bit about the types of data that you're tracking, maybe when, how often. Happy to hear more on that, as well as uh maybe the tools, the the approaches that you're using, and the ways that you then leverage the data that you are collecting into actionable insights.

SPEAKER_04 27:45

So, as we're kind of building this plane that we're flying here, I think reflection is really helpful. Look at your program's priorities. What are your mission, vision, values? Uh, what are your competencies? What are your institutional goals? You don't have to track everything, but what are what start to track what aligns with your goals? You know, we are planning to use a combination of lots of things to help with this. Learning management systems, exam software, clinical education platforms to help us manage and collect and secure, keep this data secure. That is our initial plan. If we can begin to collect authentic data or make it intentional to review it and report it to these appropriate stakeholders with that transparency that that will hopefully rebuild that culture of assessment in our program.

SPEAKER_00 28:40

Yeah, I couldn't agree more. And I'm actually very jealous that Rebecca gets to build the plane and then fly it. Um, she's in a in a much different position than coming into a program that's been established and having to tweak things along the way. Um, I will say this is kind of a million-dollar question. And I will put in a plug for our new community on the ACAP website. So we have an assessment community. So please join. Um, and that kind of got up and going because I'm a big LinkedIn user. So I put my new position on LinkedIn and I actually had probably 11 different programs reach out to me and say, what exactly is your role and what are you doing? Because we're really interested in starting this or having somebody kind of lead our assessment because again, it it is a lot. So kind of going off of that, I contacted, I met with all of these programs. I contacted ACAPT and I was like, what can we do? Because this role varies so much and people are like, this is so much data. Where do I start? How do I organize this? What tools should I use? So again, it's it's the million-dollar question. But as Rebecca said, it's kind of what are your end goals and start there and go backwards. And how do I need to assess them from there? And I think about it just like if you have a weight loss goal, like I want to lose 20 pounds, that's your end goal. How are you gonna get there from there and build it backwards? Um, so yes, join our ACAP community on assessment and we'll all post questions and resources so we can work together to create some good assessment tools and strategies going forward.

SPEAKER_02 30:16

I don't have much more to add because I agree there's lots of different, I mean, our I think just even in our conversations the three of us have had, and I've had with a few other people in different programs, everyone's doing things differently. Um and I think that's probably because there's lots of different approaches. And I think that's where the community is going to be great to hear different ideas to help problem solve together, brainstorm together, and celebrate each other, also, hopefully. Um, because this is also a really tough role to be in, tough area to be in. And we should celebrate that. But I would also say I think this idea of building the ship backwards, you know, starting with the end in mind, is really important. When you're in a program like Ashley and I are in are in that is established and has a long track record of different assessment approaches, different ways of gathering data, different um mission statements, strategic plan initiatives, um, different curricula over the years, you can find, I think, that it's important to really be careful and intentional about what you are collecting. And it requires a review all the time to consistently review your processes to ensure that they are aligning with what the current mission, vision, strategic goals are for your program, what your strategic plan is, what your program outcomes are. Um, again, beyond licensure rate and graduation rate, but what is your specific program's priorities, unique identities? What kind of graduates are you preparing? Um, and that should be reflected, I think, in your program outcomes and your curriculum and in your curricular outcomes. And that's hopefully what you're collecting. But it does require consistent constant revision and review of your assessment methods across from admissions, you know, from admissions to graduation and everything in between and all components of your student experience, your faculty experience. And again, I think that's why having one person who is sort of overseeing all of those pieces is really helpful. And that's the benefit of that and the challenge when you have more of a distributed model because there are just so many pieces to it.

SPEAKER_03 32:45

So, Ashley, on that, uh, in sort of follow-up, your curriculum committee involves more than the DPT program. And I think you're not alone in that. Programs have accompanying uh residencies, fellowships, PhD programs, or again, programs sort of along the continuum. I'm curious if you can speak to uh even higher level, not only integrating with your programs, uh integrating your assessment with your programs mission, but also all the way up to the institution and the other sort of adjacent and complementary programs that you're involved with.

SPEAKER_00 33:18

Yeah, that's a great question. So this is actually new for UAEB as well, and we're doing a lot of different assessment at the university level. So right now, my role in my department with other program directors, um, we've really focused more on our institutional assessment first and foremost, because we have to turn in like our yearly student learning outcomes and we get programmatic reviews and things like that. So we've kind of focused more early on in that institutional um arena, if if you want to call it that. So we're making sure we're kind of aligned with the university first and then we're kind of taking it and stepping it back. So one thing that we are planning on doing, we have a strategic planning retreat every year, but we are working together to create a framework for presenting our assessment data. So every program will have a similar presentation. So we are going to do like admissions or student outcomes, all of that. So we collectively look similar. And I think that's a really good approach because we do have vastly different programs, and it's hard to it's hard to make that cohesive when we all have different things that we're aiming for. Yeah, that is an added challenge on top of the challenges and opportunities already mentioned. And I was gonna add one more thing that that kind of goes along with that as well is faculty workload. So within an established program, like Katie was saying, this is kind of an iterative process. So if you're changing things and adding things, it is a challenge with faculty workload and adding things to them and and moving things around. So I think that's a big consideration too when you're when you're setting up your assessment process. Great point.

SPEAKER_03 34:59

And Rebecca, I'm curious for newer programs or even established programs who are newer to some intentional structured assessment or revisiting how they do things, this can feel really overwhelming. So if a program is starting from scratch or is thinking about reimagining, do you have recommendations on where to begin?

SPEAKER_04 35:18

Yeah, you are correct. It is overwhelming. Um, I think one of the biggest benefits was assistance from a program director to connect me with other people because oftentimes you don't know who is in the space. Not everybody has a director of assessment on their profile and just making connections and then just beginning conversations. And I think that's helpful because then you can start to see the different program aspects of who's collecting what and their type of personality and their program and what they're pulling in for from assessment. And then go back to your program. What are your mission, vision, or values, and program goals? Start with uh that a simple CAP D standard, one A and one B. Start there and then slowly start to build your assessment profile for your program and what that looks like for you.

SPEAKER_03 36:09

That's great. And we've we've danced around it a little bit, but let's ask the very explicit question to compare your three models. What type of workload allocations do each of you have in your role, or what thoughts do you have on that, given the overlap of how many things we do as faculty in our programs?

SPEAKER_00 36:26

That's another great question. So I highly recommend director of assessment, or if you're doing assessment processes, to have dedicated workload. For programs that I've I've met with and talked to, this workload ranges anywhere from 20 to 40%. So I think that's a conversation if you were to get in this role that you need to have with your chair or program director in terms of how much time you think it is going to take up of your workload.

SPEAKER_02 36:54

Yeah, I'll share. I I don't I don't have a director of assessment position, but assessment curricular assessment is part of my racked into my director of curriculum position. And so I have 25% protected administrative workload essentially for my director of curriculum position. And again, that encompasses anything related to curricular assessment, which is part of that role.

SPEAKER_04 37:22

Yeah, Katie, likewise. I I have a certain percentage dedicated. I think that's also what's helpful when you do have a dedicated assessment person is to be able to just justify giving them that time um to uh to dedicate towards assessment and and the energy to to create that as you know, that assessment personality in your program.

SPEAKER_02 37:44

I was just thinking about how you would quantify this role, which would be just is so difficult, I think. So if you are in a position of trying to advocate for the role or advocate for workload, some of the pieces that I think are really important to consider are the time it takes to collaborate, the time it takes not just to, you know, look at data and look at and analyze, but as we've been talking about this whole time, this position is connected to so many different aspects of the program or programs, especially in Ashley's uh position. But you are interfacing with so many people, which takes a lot of time. And I think ensuring that that's part of the conversation when workload is discussed, it's very difficult to quantify. I workload is difficult to quantify no matter what. That's a whole probably a whole other podcast episode. But I think if nothing else from this conversation, if someone has never thought of the director of assessment position or is starting to dip their toe into considering what that looks like in their program, I hope that they're hearing how much collaboration it requires and time it takes to integrate meaningfully all aspects of assessment throughout the program. And putting a number on that is difficult, but I would advocate for a higher than lower if possible.

SPEAKER_03 39:17

And for everyone, what are your thoughts on the balance between faculty workload and responsibility for assessments versus administrative staff support? I can imagine that programs look very different in that balance as well.

SPEAKER_02 39:30

I would not be able to be director of curriculum without the staff support. I would just say that. So I can translate to director of assessment. I can't imagine not having some staff support there. Um it there is definitely a need to clearly define roles and responsibilities within, you know, if you have administrative support. But I think that the aspect of, you know, depending on what system you use, you know, if you're using a clinical education platform, if you're using ExamSoft, you're using Qualtrics, even, you know, having someone who is managing those databases and creating and um collecting through them, that that's a huge aspect of this. And having administrative support, I think, allows you the faculty to then have more time to evaluate and to strategize and then and to collaborate if they're not doing some of the like building of the frameworks and building up the infrastructure and and supporting that infrastructure.

SPEAKER_04 40:39

It's so difficult. This is where the it depends comes in because you you don't know how much collaboration it's going to take, and you don't know how much planning is involved. And you're you're you're talking about so many different programs out there. You've got cohorts of 40 and cohorts of 100. And it's so difficult. But I think um just being able to start somewhere with assessment and then keeping uh detailed notes so that you can then have those conversations of any adjustments that need to be made, any help. I I I know that right now I'm I'm preparing um as we integrate assessment into everything we do, I'm preparing um my portion of our faculty retreat. And one of the uh slides that I have is titled Why I Need You, because you do need everybody to make this thing work. So um I think from my aspect, it's really very much TBD. Absolutely fair.

SPEAKER_03 41:42

And so we've talked about again a number of different examples across uh different programs, approaches, and different stages of where programs are at. Do each of you have thoughts on resources or tools that are currently missing in the assessment space or I do.

SPEAKER_04 41:58

Very much so. Thank you, Sarah. I think, you know, one thing for me is that I and I know it's different it's difficult because every program has different goals, but it would be very helpful to have like just a big picture framework, like a mind map almost, so that you can just go from space to space going, no, we don't have that. Check that off. So then I need to go here. And I think it's not a prescriptive checklist, but some sort of structured guide for assessment and PT education would be really, really helpful.

SPEAKER_00 42:27

We should create that in our community. Like, here's admissions, what tools can you use to assess admissions? I love that idea. I'd say the one thing I feel is missing are leadership tools. And this is kind of a hot topic in healthcare, and evidence is really emerging in this space about how to assess leadership, which is important because we do want to produce leaders in the field to take over when we all retire, right? Uh so there is a validated framework, the leadership competency framework, to help kind of guide development and assessment of potential competencies in this area, which we don't have that leadership competency currently in some of our um our CBE frameworks. And there's a team that's developing some surveys to assess leadership. So I'll give a shout out to Jennifer Green Wilson and her team, um, Diane Clark Donline, who are working in this space. So hopefully we'll have some of those tools out soon. I think they're aiming for about 2027 for a release.

SPEAKER_03 43:23

I'll use the opportunity to also uplift any resources that ACAPT has to offer, considering things like the Blueprint, the Academic Blueprint, the ACAPT Excellence Framework, the Leadership Compass. So for listeners who have not explored in-depth those resources available through ACAPT, be sure to log in, create a profile if you haven't done so. And as member beneficiaries for institutional members, you'll see a really wide variety of resources to get you started as well, in addition to the great suggestions that our speakers have today. So, where do you all think that programs struggle the most then? It seems like a very parallel. Question to what we've just asked. Where do you start? What framework do you follow? How do you know how to advocate? How much time will you need? What staff support is needed? Is there anything from a challenging standpoint that we haven't talked about that in each of your roles you see as a challenge that might benefit others to hear about?

SPEAKER_04 44:19

Yeah. What are you going to do with the results? How is that going to be used? For us, it'll be an ongoing discussion. I would love to hear from Katie about what when you get this data and you emerge with some of these trends, what do you do with that?

SPEAKER_02 44:37

That's a great question, Rebecca. I don't have a great answer to that. I you hope that it results in change, but I think what this the struggle can be is that evaluation piece that we talked about at the beginning of the conversation of truly what do you what does it mean and what do you do with it? And is everyone on the same page with what it means and what the priorities are? Um, you know, you can get a lot of data and you can have a lot of different things that emerge as needs or opportunities, but we can't do everything, right? So I think one of the areas that I struggle with, I think probably others do as well, is how do you prioritize? How do you decide what is the most important, what's feasible, you know, what's the most impactful. And I really think that's where even if you are, you know, someone in a director of assessment position, that's really where the importance of collaboration and clear alignment with every everyone in leadership positions in your program come into play, where you are really all on the same page with identifying what priority, what's what is a priority and what to do with that information. Because again, we can't do it all. Uh, we are, as Sarah, you said earlier, we're faculty and we're all faculty, we're all stretched lean. And academia overall is a slow-moving beast. And so when you do want to make change, it requires a lot of work. It requires a lot of time to be intentional about it. So I think that's, you know, back to the original question of where we struggle most, I would say that would be where I struggle the most. I think, and just in terms of how do you decide the what next when you have maybe 10 different options in front of you? And what do you do with that?

SPEAKER_00 46:41

Yeah, I'll echo what has already been said, but really the management piece of it and how much data it is and what to collect and where to go from there, but also the faculty buy-in. And I know we touched on this a little bit, but um, especially if you've done things a certain way for a very long time and now you're changing processes and evaluating different things, um, it can be difficult to get faculty buy-in. So I think Rebecca mentioned this, but just always mentioning the why. So why we collected this, why we're looking at this, why we're disseminating it out, um, and what is our end goal and always to keep that in mind.

SPEAKER_02 47:18

I think that relates to accountability too. You know, if you are all on the same page, you're moving in the same direction, all faculty, all leadership, that when there is opportunity for change identified through data, you know, maybe through course evals, through student feedback, through program outcomes, whatever it is, um, that there is a sense of accountability from everyone who's impacted by that, who says, okay, this may have been how I've been doing it forever, but clearly the data is showing me time and time again that we need to change something. And that change is uncomfortable and no one likes to change. And it takes a lot of effort and time to change the way that you've done things for years and years, or what you feel that you've finally settled into how you want to teach your course, or what content you want to include, or um whatever, whatever it is. And um, and that can be a really hard thing to accept. But also if everyone is on the same page and you have that buy-in, you have that culture, Ashley. Like you mentioned earlier, uh you hope that it the adjustment isn't quite as painful.

SPEAKER_00 48:36

And to play devil's advocate about that a little bit, change is hard. But then on the flip side, you sometimes have people that want to make quick changes. And that's something I've been trying to emphasize a lot in my role is just because you have a off year or one piece of data is showing this, it doesn't mean you have to flip and do changes. And you want to see trends in data and over several years of time. And you want to see that in various pieces of your assessment data. So just because one thing is showing something doesn't mean it's seen in different evaluations in the future or assessments in the future. So just kind of keeping in mind we're looking at trends and data overall, and we're looking at a bulk of data.

SPEAKER_02 49:20

If we were on a Zoom right now, you'd have the hand clap thing going and the tape, because I totally agree with that statement, Ashley.

SPEAKER_03 49:28

Yeah, that's a that's a really great point. And we see curricular creep, we see a challenge then in having that central understanding of what is happening from an assessment standpoint. And that that can be a challenge for sure. And we've talked about how faculty see the explicit portions of the assessment process when it involves them, or maybe when there's a large report put out, and assessment also lives behind the scenes quite a bit. There's a lot that general faculty, not in this space, may not know. So just appreciating what's been brought forward today, because the impact is really anything but small. Um, so as we're getting near the end of our time, uh, what trends do you see shaping the future of assessment in DPT education?

SPEAKER_04 50:12

Yeah, I I hope to see a more longitudinal assessment plan where really the focus is early identification of students that need assistance. And if you're not collecting data on those students, you're not gonna know. Um any student that might need support academically or otherwise. Um and then just more emphasis on faculty development involvement in assessment, like that, like we keep talking about, you know, the assessment culture uh instead of just data dumped towards the end when CAFT is coming, you know. Um we want to I I would love to see it brought forth more in terms of what we do on a day-to-day basis and not just something we're trying to collect at the end for some sort of um information dump.

SPEAKER_00 50:58

I'd like to see us work more collaboratively. So we even talked about this assessment would be a great pre-con in the future for ELC and just walking through all the different types of assessment and what you can assess. Um, because that's that's kind of hard. You don't know what you don't know, right? So meeting all of these different people have roles in assessment and different aspects of it has been incredibly helpful for me. So I just think if we had this nice community where we're sharing resources and talking to one another and saying, hey, we're seeing this in our data, are you seeing this is helpful because not only is it important to look at trends at your institution, but nationally as well. And that's something we didn't we didn't talk about earlier. But I mean, just because you're seeing, like, for instance, your admissions um numbers going down, it's important to look at that nationally as well and seeing if that's a trend there. Um, so I I just overall I'd like us like to see assessment be more collaborative amongst all of our institutions.

SPEAKER_02 51:57

Yeah, I don't have much more to add. I actually was going to say something similar to Ashley. I really have appreciated the recent connections I've made with individuals that are in director of curriculum positions or in director of assessment positions. As somebody who is also in the clin ed space, anyone who's been a DCE knows that there is a really built-in community there of support and networking and collaboration. And it's been really interesting to identify the same need in these spaces where you can really benefit from learning from others and also really recognizing, I think, a gap in the preparation for people who are taking on leadership positions like this, um, or any leadership position that includes assessment as a component of it, whether they are director of assessment or director of curriculum or program, any of those positions, um, really having a clear framework from which to start to understand that process. I agree with Ashley, you know, pre-con educational opportunities, um, whatever that looks like, I think there's great opportunities for development and for emphasis to Rebecca's point on the longitudinal, the importance of collecting at the beginning, of starting at the beginning with the end in mind, collecting early with the end in mind. Um, and and I'm not sure everyone is prepared to do that or really has a great framework to do that well.

SPEAKER_03 53:27

And the resources as ACAPT continues to evolve and grow in its second 10 years, thinking about the institutional profile survey, um, we are certainly uh still in a space with a lot of room to grow, but the ability to submit your data uh each year and see the benchmarking, pull your reports after you submit, take a look at where you are compared to others, not to judge yourself, but truly as that self-assessment perspective, you in and of yourself. Ashley's point about uh trends in admissions, for example, are others experiencing this? So the community, the national data that uh is growing as we develop in a capta culture of communication, I think all of these things go hand in hand. And future offerings for developing programs, there's there's a lot coming. And so sharing those needs and being explicit and collaborating with each other, we get a lot of really fantastic grassroots efforts coming forward. All right, in closing, you all get to go into the hot seat for a moment. A rapid fire question that I want each of you to answer. Assessment becomes meaningful when?

SPEAKER_02 54:32

When it is clearly aligned with meaningful outcomes and not just requirements and results in meaningful change.

SPEAKER_04 54:44

When it's intentionally planned, transparent, and most importantly, support student growth.

SPEAKER_00 54:51

When it informs constructive change and opportunity.

SPEAKER_03 54:55

Fantastic. Thank you so much to our outstanding colleagues on the call today. I have learned a lot. I'm sure we all have a lot to continue discussing. Emily, I will turn it over to you. Thank you.

SPEAKER_01 55:05

Wow, this ending is going to sound repetitive, but assessment isn't just about collecting data. It's about using it to tell the story of your program, guide decisions, and improve outcomes for students. Thank you to Drs. Katie Myers, Rebecca Moore, and Ashley Parrish for sharing your expertise and giving us a clearer picture of how strong assessment practices support strong programs. For more conversations like this, visit ACAP.org and stay connected through FlexCast. You can also join the assessments community and further the conversation at www.community.ac.org. Thanks for listening. Until next time.