Have you ever wondered if you could control the cursor on your screen with just the movement of your eye? Students in the Applied Software Engineering Program are doing just that. NKU Associate Professor Nicholas Caporusso and students Trung Cao and Brett Thaman join host Mike Nitardy to share about the Applied Software Engineering Program, and their new eye tracking technology.
Learn more about the Applied Software Engineering Program.
Have you ever wondered if you could control the cursor on your screen with just the movement of your eye? Students in the Applied Software Engineering Program are doing just that. NKU Associate Professor Nicholas Caporusso and students Trung Cao and Brett Thaman join host Mike Nitardy to share about the Applied Software Engineering Program, and their new eye tracking technology.
Learn more about the Applied Software Engineering Program.
Welcome back to the Informatics Café. I'm Mike Nitardy, your host and with us today in the cafe, I'm pleased to have Dr. Nicholas Caprorusso. He is the Assistant Professor of Computer Science here at the College of Informatics at NKU. He's also the Co-Director of the new applied software engineering program here at NKU. Nicholas, welcome. It's great to have you.
Nicolas Caporusso:Hi, Mike. It's a pleasure to be here, and hi to all of the listeners.
Mike Nitardy:Well, also, why don't you introduce we've got a couple of guests with us today here in the cafe as well.
Nicolas Caporusso:Yeah. So this is a great opportunity for me to bring here two of my best students who are working with me on a project that we're going to talk about probably today. Brett Thurman, and Trung Cao.
Brett Thaman:Hi.
Truong Cao:Hi.
Mike Nitardy:Well awesome. It's great to have you guys in the cafe here today. What years are you here at NKU?
Brett Thaman:I'm a senior. So I'm graduating in May.
Mike Nitardy:Awesome. Well, congratulations. And how about yourself?
Truong Cao:So yeah, I'm a junior at NKU. And I'm doing double major in computer science and applied software engineering.
Mike Nitardy:That is great. And Brett, Trung, I'd like to ask you a little bit about your backgrounds where you went to high school how you found your way here to, to NKU.
Brett Thaman:Yeah, so I grew up in Hebron, Kentucky, which is pretty close to here, about 30 minutes away. I went to Conner high school, I graduated actually 2014. This is actually my second time coming to NKU. So when I graduated, I thought I wanted to do music, because I was really good at it. And I knew that I didn't want to do music performance, because that seems to be really hard to get a steady job. As cool as it would be. So I decided on music education. And I did that degree for about three years. And I just, the longer it went on, the more I knew I just didn't want to do that it got to the point where I was like, I'm just wasting my time here. And I just couldn't do it anymore. So I just stopped. I ended up just dropping out. Just started working like it UPS like at a warehouse just, you know, throwing boxes in trucks and stuff like that. And, you know, it just I hated that too. And I was like, There's no way I want to do this for the rest of my life. Like I got to go back to school do something. And my friend had actually done a music degree at NKU. And then he went to I think Cincy State and got his some not a computer science degree, but something similar to that where he was programming. And he'd enjoyed that a lot. So I was like, Well, you know, I'll try programming. So at first I went to Cincy State and I was like, seeing the degrees they had there, they had an associate's degree. And I was like, Well, I feel like I should just, you know, get the bachelors because I feel it's more safe, it's probably easier to get a job. So I decided to go back to NKU. Ended up I just went to I think I had a meeting with there's like an adult learners services or something like that at NKU, which helps, you know, students who haven't graduated, come back to NKU, and I went to them, and I just asked them like, alright, what, what degrees can I do? And it was like, well, with computers, it's either you can do computer science, which is like a ton of extreme math. And I was like, I'm not a math person.
Mike Nitardy:Pass
Brett Thaman:And then the other option was CIT at the time. With in, they had different tracks for that. And I was like, Well, that sounds pretty good. So I decided to go with that. And that's that's kind of how I got into it. I will say I wish the the ASE, the applied software engineering degree was was there when I when I came there, because that's exactly what I wanted to do.
Mike Nitardy:Right
Brett Thaman:The degree I'm doing now is more focused on like administration, so like, you know, handling like computer servers and stuff like that. But the applied software engineering, like the research project we're working on is really, like satisfying to like, make something you're like actually creating something.
Nicolas Caporusso:Yes.
Brett Thaman:Like, that's mine. I made that.
Mike Nitardy:That is awesome. Yeah, that's awesome. Trung about yourself.
Truong Cao:So I am, I'm not originally from here, to be honest. Like I'm from Vietnam, I'm aninternational student. I went to Whitman came High School for the gifted, and I specialize in computers, like IT, and I after I graduated high school, I took a gap year. And I like to try something new, like I want to study abroad. So I spend that one year to learn more about like computer science, and then also I prepare for some certificates and stuff so that I can apply for NKU. And luckily, they gave me a lot of scholarships, so I decided to come here and then enroll in NKU like in computer science.
Mike Nitardy:Awesome. That is terrific. What's great to have you both Nicholas, it's great to have you here. This is going to be a fascinating conversation for our listeners. We're really going to be exploring a little bit today about the new program applied software engineering, and also talking about human and computer interaction and maybe some studies and work that you're doing in that area. Why don't you start off Nicholas by kind of explaining what applied software engineering is. And then also, maybe we can dive into the HCI component?
Unknown:Sure. So the applied software engineering is our newly launched program here in the Department of Computer Science. And it started in 2020, with the idea of preparing students for the needs of the industry, and companies demand students who are ready to work on real life projects, and the Apply software engineering degree has been designed in order to give the students, to expose the students to tools, methods, practices, and experiences that help them get a job and perform from day one, basically.
Mike Nitardy:Well, that's fantastic. So in the past, was there another area or field that people would get degrees in to start doing, you know, software engineering, computer science, perhaps or . . .
Unknown:So historically, computer science would be the degree that you would get to then move to software engineering, the, ah, issue with computer science is that in the limited duration of a degree program, you cannot explore both the foundations of computer science and computing in general, and applications. So you have to focus on one and clearly the most important part in this case would be the foundation, having students understand what computing is, what an algorithm is, and how to make it work and how to improve it. But then students will lack that kind of practical experience, that tangible expertise that will help them realize applications and work for companies and, and they would lack that initial talent that company would would need in order to hire them. So what we're trying to do with applied software engineering instead is shifting the focus on rather than just computing what it needs to what it takes to create applications. So learning how to think about the design, think about what's the problem, and ways in which that problem can be solved. And then using tools, programming languages, frameworks that are out there, in the best way possible, so that we can develop quality software by reducing the required resources being that time or cost.
Mike Nitardy:Interesting. Interesting. So would it be possible to give a little bit of a overview of the difference between the types of courses, a computer science major might take and an applied Software Engineering major might take?
Nicolas Caporusso:Sure. So the computer science degree is heavily focused on, as I've said, the history and theory of computation. So, and of course, computation has a foundation in math. So math classes would be the starting point to get exposed to concepts in computer science. But then, students would learn algorithms and data structures and operating systems in courses that help them expand that knowledge about the computer and how computers work under the hood. In apply software engineering, instead, we start exposing students to how to build applications from the beginning of the program, we start with a course called Full Stack application development, that builds up on what students learn in the Introduction to Web Development course. So after students learn how to create a website, now we have them think about how a website can become an application. Okay? Most of the applications that we use nowadays, Spotify, Instagram, they are available on the web. So the web has become a fully fledged environment where we can do pretty much anything. And the same things that we can do on a desktop computer can be done in the browser environment.
Mike Nitardy:Right, right.
Nicolas Caporusso:And so we help them understand how they can create a an application that not only runs in the browser, but communicates with the world uses databases, and it's interactive and interoperable with other types of applications. So for instance, we could build a, an application that connects to Spotify gets our favorite playlists and enables us to store the songs or change them and edit them in a more fashionable way.
Mike Nitardy:Wow.
Nicolas Caporusso:After full stack application development will help them understand what happens in the so called server environment. So it's basically understanding how the engine of systems like Facebook or Google works. So what happens if we need 1000s of users to retrieve and update information, all the technologies related to that kind of environment, the server side. And then there are several other courses that teach students how to design a user interface that users will love, okay. And this is the case of human computer interaction. And most of the other courses focus on software engineering. So how to think about the design aspect, how to optimize a, an application, how to make it more efficient, how to reduce bugs, and prevent bugs from happening. And finally, students develop a project in
Mike Nitardy:Okay. their capstone course, where they decide what they want to do, and apply all their skills. In addition to exposing them to tools and methods, our focus is also on having them create a portfolio, it's really important for companies when they hire students to be able to see what they've done in the past. So what's on the paper, you know, matters, but what they know, is more interested in what was demonstrated they could do. So having a portfolio of works, real world applications that they've built in the past really helps them showcase their talents. Fantastic. So I know it's when did When did all this started? NKU this this specific? It's is it its own degree program? Or is its own? Major? Or what is it exactly?
Nicolas Caporusso:It's a new degree program, um, which would be a major, and it started in 2020. So we this is now the second year, we'll start to see our students graduate next year, and in 2024. And we are really excited about that, we'll start to see the results of our work. But students from the ASE program are already getting internships and a Trung is an example. But there are way more out there. Currently, the enrollment is about 30 students. And it's great for a program that was just launched the last year, and we're growing in the next years.
Mike Nitardy:So I guess obviously, the, one of the reasons for this program is because the demand is out there, like you said earlier for people to be able to do the work in but when you said software engineering to be able to do that, is that the same thing as coding, or is there a difference?
Nicolas Caporusso:Right, so I'll get to the first part of your question that demand Before launching this program, we actually consulted with Sure. students and with industry leaders who are part of our advisory board, and both sides expressed their interest in something that would tackle the state of the art technology and kind of fill that gap between what computer science historically has been and programming languages such as Java and Python, and then more modern frameworks that use JavaScript or Node.js or React to enable that kind of rapid application development. So and then, when it comes to your second question, the difference between software engineering and coding, usually, software engineers are seen as code monkeys.
Mike Nitardy:Okay.
Nicolas Caporusso:And that's what we want to steer the perspective from. Okay, our idea of software engineering is somebody who was able to understand a problem and design a software solution before you know getting to coding, understanding what are the main components, where risk is in assembling these components, and then selecting for each of these components, what's the best strategy? Sometimes coding is not the best strategy, but instead acquiring a new technology that's, or a technology that's already out there. Incorporating a tool that already exists helps us avoid reinventing the wheel, optimize costs, and also reduce the workload and ship a solution faster.
Mike Nitardy:Definitely. Well, let's talk a little bit about this in practice than. Obviously we've got two students here. And you also mentioned HCI. How does that all work together? How does that all interplay?
Nicolas Caporusso:Right? So human computer interaction is one of the courses that are part of the applied software engineering program. And to be honest, is the course that I loved loved the most because it's part of my background and experience. Before joining NKU I've worked in the HCI field for several years first studying brain computer interface, and then studying systems that robotic systems that would interact with humans in the context of nuclear decommissioning, so we can assist humans to perform a task faster? Or how can we avoid sending humans into a nuclear facility? Right, right. We need to remove some uranium buyers. We don't want humans,
Mike Nitardy:That's not a high stress level job at all.
Nicolas Caporusso:Right, exactly, but it's even less stressful if we could design machines that helps individuals move faster, more safely, and, and probably, you know, control the machine without being there. Okay. So, when I've joined NKU, I've started sharing this passion of mine with students and, and in in fall 2021, we've started working on a new project that aims at using eye tracking for improving the way users interact with computers and desktop computers specifically. So the initial idea was to see whether we can capture more data from daily basis interaction with computers, we spend many, many hours as developers but also as professionals in front of a computer. And getting access to eye tracking data where the user is looking how much they're focusing or how long they're focusing on the different components of a user interface will help understand whether they are using a an application effectively, whether they're tired, whether they are attentive to, for instance, a specific portion of a website. So we were thinking about that part, and then Brett and Trung joined the team, and the project then completely shifted and expanded into something, I would say, bigger and even more interesting.
Mike Nitardy:Well, let's talk about that. So why don't you set the stage for us because now you've you've, I'm very curious and eager to get into this and hear what this is about?
Nicolas Caporusso:Sure. Um, the project started as a pure research initiative, with the idea of understanding, for instance, whether current eye tracking technology could be made available to users in a more effective way, could be made more affordable. Because the problem with eye tracking right now is that the majority of eye tracking applications require the use of an external device, it's not only the price that make it less available to users. But it's the fact that users have to adopt some new piece of technology and bring it with them as they need to use this device. It's like if notebooks didn't have a keyboard incorporated in the laptop, you would have to carry an external notebook, which would be an inconvenience, and probably would prevent you from typing. Right, right. So our initial idea was to be able to instead of using external devices, external eye trackers to use the webcam only, okay, in order to detect the location of the user's gaze, so where they're looking on the screen, and then acquire that data. And then we've started thinking about ways in which we can use this information, and maybe build our own system that improves what's being done today.
Mike Nitardy:Hmm. Okay, so where did these two come in?
Nicolas Caporusso:Well, I'll let them speak.
Mike Nitardy:Yeah, go right ahead.
Brett Thaman:So I met Dr. Caporusso, in a server side programming class. And I remember one day, he just mentioned that he was working on research, and I had already done a undergraduate research project, the last summer, and I'd really enjoyed the experience and learned a lot. So I just decided to, I just started to go into his office and ask him, like, can I help you on this research? Like, what are you doing?
Mike Nitardy:Nice
Brett Thaman:I like the he he laid out the projects that he were he was doing. And I didn't really have much experience in the stuff that he was working on. But he just kind of like accepted me on. And I've been really working hard to learn a lot. And it's been really rewarding. And I've learned a ton working with him.
Mike Nitardy:Is there anything in particular that you're working on that is?
Brett Thaman:So a lot of what I learned was using JavaScript, which is something I really wanted to learn, because my degree is CIT, with an emphasis in web development. So I hadn't had much experience with that before and our project is heavily based in that language.
Mike Nitardy:Okay
Brett Thaman:So there was a lot we did a lot of work. I think the first thing I did was we we made a mask program. So we wanted to test whether It had the effect of a user wearing a mask when using the eye tracking product, like, would that really affect, like, where the user is looking? So I made a, like a little code, program that like,
Mike Nitardy:Because it's probably prepared proprietary.
Brett Thaman:Yeah, no, it was just for like, collecting data Yea, no, I'm just joking. and stuff. So we just wanted to collect a bunch of data, have a bunch of students come in, you know, wear masks, then don't wear masks and like test whether or not you know, just the effect on it, if it affects the accuracy. So that I did that. And then currently I'm working on, we're trying to see if so there's these little computers called Raspberry Pi's. And you can like have little cameras hooked up to him, they're pretty cheap for you know what they are like little tiny computers, we're seeing, like, if you had a classroom setting, and we're doing this big eye tracking setup, if it would be easier instead of having to, basically because the, the eye tracking is really intensive on the computer. So if everyone, you know, if you're doing it on the computer, it might slow down the So we could have a system where it's like, we have each computers. Sure, yeah Raspberry Pi with a camera, and they're sending the data. They're either, either maybe we're trying to get it to where we can have the Raspberry Pi's just doing it, the computing so that that way the user's computer isn't like, you know, like slowing down, like that. But, and then the other option is like the Raspberry Pi's all send the video to a central server in the room. So a bigger computer that has the processing power to handle everything,
Mike Nitardy:Okay
Brett Thaman:Runs all the processing, and then sends back the data to the user's computer. And then they just have all they have to do is like, they know where the points are, they know where basically they know where the person is looking. That's all it's sending back.
Mike Nitardy:Interesting.
Brett Thaman:Yeah.
Mike Nitardy:Interesting. How about yourself, how did you get involved in all this?
Truong Cao:Yep. So, I first heard about this project was like, when I was in his class for server side programming. And yeah, he briefly introduced that to me. And I was like, that sounds interesting. So I went to his office, and then we have a discussion about, like, how we gonna plan for this project. And at that time, I didn't have like, a lot of experience in, like, doing full stack development. Like, I know, I have knowledge, and like foundation in like front end development and back end development, but like, I don't know, like, how to connect them together. Interesting. Yeah. And then like, so I was, like, that's very interesting. And I think that that would benefit me, like, in my career path. So I would ask him to join and like, and, yes, and he has accepted that and then during that time, I have, I did a lot of research. And I have, like, self taught myself, like, because there's a lot of new technology that we are adopting, and we are using to make, like a reliable and, you know, like, to make more to make it more secure. And currently, I'm working on you know, like, we planning to create, like, a eye tracking, like software development kit that
Mike Nitardy:Oh, wow.
Truong Cao:Like, implement and like, you know, like, develop, like, an incorporated like on top of some, like remote collaboration app like Zoom, Microsoft,
Mike Nitardy:Interesting
Truong Cao:Something like that. And also, we we can, currently still working on creating some apps that, you know, like, incorporate eye tracking, and like our research so that we can bring it to user and possibly to the market.
Mike Nitardy:That's fantastic. That's fantastic. So, so are these the only two students that are working with me or imagine there, there are several other ones Nicholas.
Nicolas Caporusso:We are expanding the team right now. Because the way it started, was just as they mentioned, the conversation about eye tracking and how we can make it better. And, and when you when you do research with students, especially with brilliant minds, you keep talking about what the goal is, but then you keep adding pieces. So right now we're working on so many different aspects that are all valuable from a research standpoint. So like, for instance, what happens if two people are in front of an eye tracking system? Can we discriminate between the two can we collect that flow? Or how can we use eye tracking information to optimize, for instance, image editing? Or how we can help doctors share data in a more efficient way? Or can we capture ways in which users interact with images and content. So every single project adds to the complexity, the to do list. And so the team is the team is growing. But I'd also like to mention the results that they've they've achieved. So so far, since just the fall, we've published, I think three or four papers,
Mike Nitardy:Wow.
Nicolas Caporusso:Then they've presented their posters at events here at NKU, they're going to present at the Capitol, in Kentucky, Frankfort,
Mike Nitardy:Wow, awesome guys.
Nicolas Caporusso:Next week, actually. So they're going to talk to legislators about their work. And I'm really excited about all these things that happen, you know, outside the classroom, just based on that conversation, and then help faculty, in a way, get more resources for projects, but also help students get that experience that really helps them, squeeze their brains and understand how they can after the squeeze, it expanded into, you know, another dimension.
Mike Nitardy:Well, I can tell just by sitting here talking to you, Nicholas, that you've got a passion for that. And it's to your credit, just listening to these two, bright individual stories that they both they had a common thing is that they both thought about a perceived gap in what they were able to do. And they and they came to you to try to get some experience in closing that gap. And obviously, you've helped them with that. So I think that is, to your credit, it's obvious that you've got a passion for this, for this applied engineering program. And also for helping students, that is just terrific. It's been awesome to be able to sit here and talk with you guys. And any any final thoughts before we close up here, anything that you'd want other students to know, or
Brett Thaman:I would recommend any student to try to get the undergraduate research project, there's a lot of undergraduate research projects you can apply for. And I think they're really good. They really helped me not only like learn a lot, but just understand that like, just because a problem seems really daunting, doesn't mean you can't like attack it and figure it out, and you break it down piece by piece.
Mike Nitardy:That is, that is really neat. That is really neat. How about yourself Trung?
Truong Cao:So yeah, I think everyone says, like, so cuddly, like, um, like I mentioned, I'm like, I'm majoring in ASE, the applied software engineering, and I heard a lot of student like, who's in the computer science, who seems like they kind of lost and they don't, they don't know what to do. Because in you know, like, like Dr. Caporusso, also mentioned about computer science, like, teaching us about how computer works, and the theory of computation, computation, but we didn't have a lot of chance to work on, like real world projects. So ASE is a program that is well designed, and by a lot of great professors in NKU. So I would recommend, like, if you, if anybody is like interested, they can, like have a conversation with Dr. Caporusso or Dr. Cho, who's another co founder and director of this program to learn more about this program, and I think this very like, has benefited me a lot because I because like doing double major in CS and ASE I have a chance to learn like how computer work under the hood. And also I have the chance to learn how to do software because software engineering is hard and that just like the degree. That gives me a lot of opportunities to advance in my career.
Mike Nitardy:That is That is awesome. And so it's been great to have you guys in the in the cafe with us today. I'm going to give Nicholas the final word if there's anything that you'd like to add as we close out here. Sure. I would say regardless of the major The main thing is to explore that entrepreneurial side which can be done by doing research and in this case, the project van he's trying to become something that can be commercially viable, but it all starts with students getting to know their professors interacting with them, and connecting with them. And trying to see whether there's a way, which at NKU is very, very common to expand whatever they do in the classroom, and kind of connect with the real world. The University is a great place where you can test your skills in a low risk environment and doing research with faculty is one of the best possibilities. And so I would recommend students to try that avenue. Nicholas, I want to ask you about any grant opportunities or funding of these programs? Have you gotten anything from the school or other sources?
Nicolas Caporusso:Absolutely. So none of this could have happened without the support of multiple parties here at NKU. The project has started with support from the Dean, the Institute for Student Research and Creative Activity here at NKU. The, an institute called CINSAM, which is the Center for Integrative Science and Mathematics. And all these places are what really faculty needs to start working on projects. And, and typically, that would support student work. And that's how I got Brett and Trung on the team, because otherwise that would be impossible.
Mike Nitardy:That's great. Thank you guys so much for joining us today.
Nicolas Caporusso:Thank you.
Mike Nitardy:Informatics. Café is a production of Informatics+, the outreach arm of Northern Kentucky University's College of Informatics, hosted by Mike Nitardy. Produced by Chris Brewer. Music and engineering by Aaron's Zlatkin. Recorded at the informatics audio studio in Griffin Hall.