[00:00:00] Welcome to Floating Questions, the podcast where curiosity leads, we follow, and stories unfold. My name's Ray, simply asking questions. Shall we begin?

Hi Yiming, how are you doing? Hello, I'm doing great. How are you? I'm great. Today is Saturday, and I had a great sleep, so I'm ready for this conversation. Uh, I'm sure there's a lot that we can talk about. I had, uh, an alarmless sleep as well, so she's very much ready. Okay, awesome. I know that I usually call you Amin, but I know that technically your name is Muhammad Amin.

I want to know a little bit about, like, why do you choose The main portions of the Mohammed portion, um, because it's [00:01:00] actually kind of interesting for me. I also choose to ask people call me Ray instead of my actual first name, which is Rui. So maybe Let's start there. Well, thank you for asking me this.

Actually, I appreciate it. It's because people kind of just choose what they want and they just use it, which is fine as well. So I come from Morocco, hence, you know, and it is a Muslim country. So Muhammad is a very particular name for us, the prophet name. So there's something really important to us. So typically people, and especially boys would have two components, first name, and they would always add Muhammad to it or very often.

Just because they would want their kid to have somewhat the prophet's name as well. But usually they would call the kid with the second part. So Amin for me, and the middle name is kind of Mohammed. But because kind of of this respects we are for the prophets, we always write it as Muhammad Ameen. Um, [00:02:00] so it's funny, like anything administrative, they would call me Muhammad because like they just read the first one.

But actually, nobody called me Muhammad growing up. It was always Ameen. Uh, does Ameen mean anything? Yes, it means trustworthy in Arabic. That's really interesting to know. So my first name is actually Rui, Rui is my mom's last name. And Rui technically is the character that was given to me. Usually people would take their father's last name.

In my case, that's the same, but I would prefer my mom's and my dad's last name to be my last name. I hope my dad doesn't hear this. Um, But yeah, so that's why I prefer people to just call me right. And also it's a little bit easier. Otherwise people also call me Yuri, which is a Slavic. name too. Uh, but that's not really my name.

So your parents also called you Ray growing up? Yeah, they would [00:03:00] repeat Ray like twice. So Ray Ray, usually that's how they call me. I see. So we both are living the same struggle, right? Yes. Um, we already dived into the name quite a bit. Maybe now, You can give the audience a little bit self intro. Of course.

Well, first of all, thank you so much for having me. It's a, it's a great pleasure, but also an honor. We knew each other from now three, four years, maybe four years ago, but we didn't talk for a while. So it's also kind of an occasion for me to catch up with you. I'm Amin, I come from Morocco, as I mentioned earlier, I grew up there until the end of high school in the city of France.

And then I went to France for my undergrad. And after France, I came to the US at MIT for my PhD, and I just finished my PhD actually a few months ago. Congratulations. Thank you. Thank you. Uh, it's a big [00:04:00] life chapter coming to an end. And next year I'll be joining Kellogg as an assistant professor. It's the Business School of Northwestern University.

So I'm moving to Chicago and I'm pretty excited about that. Sweet. What are you going to do between now and then? Well, I hesitated a lot whether to start directly at Kellogg or, um, or to do something else, but I thought I wanted to take a little bit of time, uh, to kind of do a little bit of a smooth transition between PhD life and professor life.

And in the meantime, I'm doing a, sort of a postdoc at MIT still, uh, it's kind of just to give me a little bit of time to reflect about my research. To have less pressure to be able to take risks, think of more ambitious ideas, uh, and have a little bit of time to take perspective and prepare for professor life somehow, because once you start all this.

projects and all the [00:05:00] research, everything goes by super fast. You don't really have time to pause and say, have a more broader perspective on what you're doing. So I think this year would be very useful for that. And it is actually, so I've been two months in the bus stuck now, and I'm really, really happy that I actually chose to do that.

Absolutely. I think it's really wise to take time in between major milestones. what is the ambitious goal that you're taking right now? In terms of research? Yes. We are in Seattle, so you can't hear us. Probably see a lot of startups, but all these startups now, most of them have this common idea of, Oh, there's NLMs.

It's great stuff. Child CPT. Let's apply it into something. Maybe someone want to start a gym, right? With NLMs involved in it. So what is the key component in these, uh, startups is given this NLM, that this transformers, these like powerful things we have, how to adapt them to a specific task. And typically how they do this [00:06:00] is by the so called fine tuning with the task specific data.

And what I want to think about is what is the right data for this task? Like broadly speaking, I would just collect some data related to this, but can I strategically do that? Kind of a more far reaching idea. Let's say I want to do a social study. Uh, like the government want to implement some policy or want to decide of some tax cuts and they want to understand what would be the impact.

So before that, they would typically run some study and they collect some data. But data collection is very expensive. You know, they need to run surveys. They need to talk to people. So again, an important question is how to strategically collect this data. What is the right people that I need to survey?

What are the populations that I need to look at to be representative enough of my population and capturing the impact of. The given decision, that's kind of the more ambitious question. I'm trying to think about this year. Fascinating. Basically you [00:07:00] touch upon like data assets and the strategy component.

And this is something I also think a lot at work, but before we get into a lot of your work and research, I actually want to talk a little bit about, Morocco, Fes, where you grew up. I traveled to Morocco eight years ago. That was my first ever solo trip and I absolutely had a blast. I loved it. I was taking an overnight train from Chefchaouen all the way to Marrakesh.

And in the morning when people get into the train people just start sharing food with each other even though they don't. No each other at all and I was so happy because I didn't have enough food with me I'm gonna stop talking about my travel because I can go on forever. But um The only place that I didn't go though is Fez because I was like jumping through Rabat and then Chefchaouen and Marrakesh and the Toubkal Mountain and then desert so I really didn't have time to go to Fez, [00:08:00] but I would really love to go So maybe you can give me a little bit like preview about what Fez is about.

What are the things that imprints in your memory? Well, my goal right now is to make you Uh, feel that you missed the best parts of Morocco during your visit. I'm usually excited when people have visited Morocco. I'm, um, quite passionate about it and its history. It kind of comes from my father. So Fas is perhaps I would say one of the most historical or the most historical city of the country.

It has quite a profound history. The fact that people don't know that much is that we have the oldest still functioning university in the world. Uh, we're proud of that. And not only that, but this university was built by a woman, which was not common back at the time. So that was in the 12th century or 8th century, I have to revise my history.

[00:09:00] But sometime around that, it's still very much there and alive today. It's somewhat of a mosque slash university. And it's a very exciting place to visit. So Fas also has the biggest walk in city in the world, in the sense that there is no roads, there is no cars. Okay, I have to go back. Well, you have an open invitation anytime.

I didn't grow up there, unfortunately, because this becomes just the historical part of the city and people live outside. But my father grew up there, so I hear a lot of stories. And it was very interesting how people Back at the time, it's not so long actually, just talking about, uh, some decades ago, every neighborhood, for example, in the town was known for something.

There is a neighborhood festivals. For example, there's a neighborhood to people who sell knives. There's a neighborhood for pottery. There's a neighborhood for leather. And in the neighborhood, you know, it would have people who work the craft and people [00:10:00] who sell at the same time. Um, and all the neighborhoods interact with each other because obviously.

Someone who makes clothes would need leather, so they go to the other neighbor. So it was a very interesting dynamics. And also everybody knew everybody, so it's somewhat safe. You know, if you have your kid that is running around, all the neighbors and not only like all the city kind of know whose kid is that.

Hospitality is a big thing. Having people over to your home, even that you don't know, is also completely normal. Uh, if someone is traveling and stopped by a fence, they don't need the hotel. People would host them at their home. My father tells me about it. I really feel like it was a great time. There was less means and resources, but there was more humanity, I would say.

In what sense? Well, there's more human interaction. Those were more warmth. So for example, you see a stranger, you know, you would not have a stranger over to your home, right? If I'm [00:11:00] traveling to Seattle, I'm not going to just knock on a random door and say, can I stay over in your house? I'm actually now reading a book about, uh, old time Syria, which is kind of similar in Damascus.

And There was a traveler from Austria who was telling about his story traveling in the Middle East, and Syria in particular, and he was saying that it just shocked him that he traveled for 30 days and did not spend a single time. Every time he stayed at people's house, they would invite her for food. So there was a lot of this back at the time, but it's somewhat unfortunately slowly disappearing.

I would say it's still much, much more in Morocco than in the USA, for example. But, uh, somewhat the new way of life, globalization kind of makes it that it's not something we emphasize on as much. Contemplate on your last point, which is the globalization or [00:12:00] the development of economic status, right? Like people just get more and more individualistic and isolated from each other because you're less dependent on each other, right?

Like the reason why. You knock on someone's door, it's because you need something potentially. You don't have a place to stay, and people are kind and welcome you to stay. But since that dependency layer is removed, then you tend to have less of that interaction. Of course, it has both good and bad sides of it.

I guess the question is, how do you retain a layer of that warmth and that interactions going forward? Yeah, exactly. I think somewhat we always see development as equivalent to economic development, but I feel like it's much more subtle than that because for example, psychological comfort and well being is not necessarily correlated 100 percent with economic development.

It's not because [00:13:00] I'm richer or more physically comfortable. that I'm actually happier. So sometimes I actually keep thinking, is humanity overall happier these last decades than they were before? I mean, obviously we have much more economic means, there is less hunger, but does it mean at the same time that we're happier?

I mean, we see all these mental health problems. Maybe they existed before, but we don't have as much data from the past, but something I feel like that we need to think about as well when thinking about developments. Yeah, for sure. Okay, well, let's maybe revert back to something a little bit more controllable.

What was your childhood interest that propelled you towards the research field that you are working within right now? Well, it's a very unpredictable path, actually. If you asked my 10 year old Amin, where would you be in 18 years, I would not have guessed I'd be here, [00:14:00] honestly. So I grew up, you know, especially in Moroccan Frances.

Not a big city by American standards and people don't have, at least at the time, that big of ambitions. I never would have imagined I would be in the U. S. one day. It seemed just so far in all sources of the world, right? So far as unreachable, just, you know, buying a flight from Morocco to the U. S. is a lifetime investment for some people, you know, like the median income in Morocco is maybe 600 or 700 a month.

So that doesn't even buy you a flight. So that was something that seemed very, very far away. I remember I was ambitious and my dream, you know, I remember saying that classroom to the teachers, I want to be president. of Morocco and change people's life. And funny enough, the first reaction would be like, well, first of all, we have a king, so you cannot be a president.[00:15:00] 

You should say prime minister. I was like, okay, fine, fine. I want to be prime minister. And that actually for a while was my dream. I was like, I want to change people's life. I want to improve. I knew Morocco was not a rich country. I knew there was a lot to do. Remember as a kid, sometimes I would walk in the street and I would see someone just looking into a trash can or something like that, I would be shocked that how can that, you know, exist and how to change that.

So that was my, my dream for a long time. But I think slowly growing up, I started realizing that politics was perhaps not the most efficient way to change things. Just learning more about the corruption that is happening, that all these politicians are, were not, at least I felt were not there to improve people's lives, but more for their own desire for power.

So in Fireland, something kind of life changing happened to me. My aspiration was kind of, I wanted to do good. I wanted to change people's lives. And [00:16:00] at the same time, what I really enjoyed in school was mathematics. I was fascinated by it. It was really my pleasure. Every time I just found the mathematical objects and the language.

So fascinating and so intellectually stimulating that I was really passionate about it. But I never really had a venue to live fully my passion. I know sounds that beyond the classes, which I had an easy top grade. there. And again, I was in one high school in Morocco. So, you know, you don't have a MIT student or whatever.

So it was not so hard, but beyond that, I couldn't do more. I remember I would go, you know, we had this national textbook of mathematics and the end of it, the last pages, they were like challenges. So I would go there and try to solve these. And that was kind of as far as I could, that was, was available to me.

And then one time I was with a cousin of mine and one. His friend was telling me about Mathematical Olympiad, and I was like, [00:17:00] huh, what's that? He said, you know, there's a national mathematical Olympiad. So you can take this national competition and be ranked among the top in the country. And after that, there's an international mathematical Olympiad.

So if you're among the top six in the country, you're sent to the national team. You can compete in that. And I was like, Oh my God, that's what I want to do. This is amazing. So I started searching online and I found the forum, uh, kind of talking about different problems and given challenges to themselves.

And that was great. Like this forum taught me a lot just from talking and I kind of could meet people all across the country. They would share books with me and I started learning and I took part in this Olympiad. the regional one in my city. And at some point I got a letter saying, Oh, you are qualified.

And then you're going to go to the national one. That was a big day for me. I was very excited because I was like, well, there's so much more than what we learned at school. And there's so much resources that I can learn from. So I started reading books. I [00:18:00] started training and that I was on what's very late because many of the people.

that I later met in this National Olympiad. I've been preparing for years because they heard about it before. So I got to the National Olympiad, we were 40 at the time, and only six of us would be in the national team. And then I talked to these guys and I was like, man, they're so strong off them. They've been reading books, they know so much.

And I know nothing about it because nobody told me, you know, nobody told me there was this. And how it works is that throughout the year, they are like, Exams and each exam, some people qualify. So we were 40, then it become 30, then 20, until we get to six. So I took the first exam. I was not at their level.

I think I was like 29 out of 30. So I couldn't make the first cuts. And then I went back and I was like, I forgot about my studies. And I just started working on that, just reading books, just trading. And I was like, I'm going to go to this international mathematical ambient. That's like my dream. So I worked.

And again, second [00:19:00] cut, I would have studied just enough to really be able to make it. And then I would be like 18 out of 20. And then I just kept going like this. And between each exam, I would study more and then until the last one. And I was fourth out of six. So I also kind of make it like every time, just enough.

And that was the Nigerian experience. And I was in the team and I went to an international mathematical Olympiad, which was in South Africa. And then my mind was, was blown away. I would meet, you know, the teams from all over the world, from France, from China, Hong Kong, the U S and these people were so, so much more advanced than me.

I still want to. best experiences in my life. I remember it was the first time I took a plane actually in my life. I was at 17 to go to South Africa. But then I also had this bittersweet feeling where this was exciting. I learned a lot. And then I also learned to be more ambitious as like these people, you know, they were talking about [00:20:00] all these top universities in the world and they are going to go there.

So, you know, why not me? Right. But then the bitter part was that I felt in Morocco, we're nowhere close to China, for example, or the U. S. or the European countries. I felt like they had so much more means, so much more tutoring. Their children, they have so much more opportunities. Then I went to France for my undergrad.

Like this international competition earns me a spot at, uh, L'Isile Winogrande, which was one of the top places in France then. Then I got into École Polytechnique. This was also one of the top engineering schools in France. And when I got there, I was like, okay, now I kind of made it, quote unquote, into a top place.

So my future was more or less assured, you know, I would have a good job. I would have a good salary. And I thought, okay, I want to do something about, you know, that opportunity is in Morocco. And that was also kind of a second big [00:21:00] change is that, you know, you could criticize the government, you could criticize a lot of things, but I felt like if I don't do something about it, meaning like us, the young people who still have these motivators, nothing will change.

If we just keep relying on other people, things will not change. So I started talking to some of my friends first, uh, around that kind of a similar experience. And I told them, Hey, I'm having this idea. How about if we do something about it? And it was a resounding, yes, let's do something about it. A lot of people have this motivation.

I started talking to Moroccans that live all over the world. Some older than me who also did Olympiad and then they were in the US, for example, in top universities. Some also in the same year as mine. Because I kind of got to know this network of Moroccans who moved into Olympia. And then I tried to start a bit of a movement to kind of change things.

And then this culminated in creating an NGO, Matémaroc. That was 2016. So now, [00:22:00] I think a lot of things changed with that. The momentum was such that a lot of people started joining. We can talk about it also in more details. The round was not easy. There was so, so, so many problems with the government. It got to the Ministry of Education.

I mean, there was a lot of issues that happened, but over time, there was a lot of momentum. We kind of created this platform where any young Moroccan who wanted to create a change now had the mean to do it. Because they could, now there's a platform, there's a network, there's people involved, we have funding, we have a lot of things going on, so anyone who contributes would be easy.

You just join, you start the projects and you do things. But back in the time, there was nothing structured, nothing where if I wanted to do change, people would do something kind of locally, they would just help people around them. They would just talk to younger students and mentor them. It was a big change because it created the platform that enabled people to More easily and more accessibly create a change.

So [00:23:00] Mathematics is still very much active today. It's eight years old. Um, they do much, much more than we did back in 2016. I was the president for three to four years. And then in the PhD, I got very busy and I thought it was the right time to pass it to the next generation. Now, four presidents passed, I think, and Mathematics is so much bigger.

I think now it has. Between 50 and 100 permanent members and a lot of volunteers, and they do a lot of things in Morocco. So it's something I'm very, very proud of. It's probably the thing that I'm most proud of. in my life so far. That is absolutely amazing. Um, I actually would want to know the challenges behind this entire process, but at the same time, I know there are so much more that we can talk about right now.

I'm just seeing a dilemma of like, where do we want to go? Um, I realized I never really [00:24:00] responded. To your questions, I remember the initial question was, how did you get to what you're doing now? And we just kind of diverted to talking about NGO and things like that. Let me actually start there quickly.

After my engineering degree in France, I was thinking a lot about whether to go to industry. I think two things kind of popped out. The first one is I want to have a positive impact. That was the motivation for many, many years life that I grew up in compared to, especially when I saw around Morocco, I was privileged.

I never had to worry about food on the table, for example. And the second thing was I wanted to do something that is intellectually stimulating to me that I enjoy. And I really struggled to find that in industry back at the time. Most people at my university would go either to finance or to consultant just because they pay more, but it did not fit these two criteria.

And I found research. [00:25:00] really spoke to me. That's why I chose a PhD. I thought, well, I could decide to do whatever I want. Technically I can use my skills in mathematics, which I enjoy to create a positive impact. So that's how it started. And then slowly I got into AI and machine learning just because I felt like this is where mathematics had the most impact.

Like most of AI machine learning, when you dig a little bit deeper, you find that it's just mathematical models of how we learn, right? We try to formally model how we learn and how machines learn. And when, whenever we say formally, the language for that is math. That's why I kind of, I got into AI. Okay.

Let's maybe wrap up this topic and go. into your research work. Correct me if I'm wrong, the two key words in your research is interpretability and robustness. Maybe let's start with just like defining what it means. So I tried to look at two questions in my PhD and they were [00:26:00] mostly influenced by my advisor, Art, who was a fantastic researcher, as I learned a lot from him.

Basically, so this is a pre NLM study, time. There's all these machine learning algorithms, there's all these AIs, but how much of it is really applied? And back in the time, and still kind of today, not much was applied. If you go to industry in general, except if it's a specific industry, you would find that They don't use much beyond like linear regression.

Actually, Ray, you would know better than me. I was in, in some industries where there's a lot of potential of AI being used, but it's not really used. And you're trying to think about why is this happening? How can we tackle it? And so there's many reasons, but the two reasons we looked at, the first one is robustness, which means some algorithms are not reliable.

For example, let's say in healthcare, we want to treat patients with an algorithm which needs to be really stable because there's people's life on the line. [00:27:00] So that's the robustness part. You want like stable, reliable machine learning algorithms give us guarantee. The algorithm should be able to tell you the performance should be at least this, and it will actually respect this threshold so you can be safe deploying your algorithm.

The second one, interpretability, is we thought ideally in practice, because humans are the ones who are responsible for the decisions, you don't want the AI to be completely autonomous in that it makes a decision by itself. And we just need to trust it. We thought we should look at it more as a recommendation tool.

And then the human make the decision, like the doctor gets a recommendation. Oh, this patient should take treatment. Hey, and then should have an explainable recommendation because of this feature and this feature and this feature and their medical history. And then the doctor would be like, Oh, given this.

Features actually make sense that I wanna prescribe this or I don't really think that's the [00:28:00] right thing to do and do something else. And that's what we mean by interpretability. I think in the industry, especially industries that are heavily regulated, including healthcare and also actually finance, uh, interpret.

is required for regulators to even understand exactly what kind of decisions that we're making, right? Like how the algorithms sort of make the risk assessment. A lot of the algorithm, not even the neural network, right? Like some ensemble tree model, is already complex enough, uh, and it poses challenges for interpretability.

What is the main framework that you are proposing? So interpretability is a very active field. A lot of people are trying to think about it, because I think people realize how important that is. And I think to this day, there is no unifying framework on how to address that, even how to define it. Actually, many people define it in different ways.

So there is really many, many things one can [00:29:00] look at. But let me give the healthcare example. Let's say I want to an AI that gives me personalized treatments for patients. One intuition one can get is if you look at how doctors actually prescribe treatments, like their medical guidelines, you will see that they're formulated somewhat like the following.

It would be if the patient has an age above 70 and they had this history of. this medical thing, and then they have a blood pressure between X and Y, then there's a treatment that you should prescribe, right? So we thought, what if the AI can learn something like that, right? You can learn a simple representation of patient depending on their features.

And once we have this representation, not only the prescribed treatments would be interpretable because the prescribed treatment would be like, if, if, and this, and this, and this, and prescribe this, but also it would describe of what the AI thinks will happen. Right. He thinks that if you give this treatment [00:30:00] that they will transition to this new Think about them as you would construct some sort of treatment groups, and you understand how patients evolve between these treatment groups, depending on the, the treatments you give them.

So you want to understand this evolution. And if you attempt at predicting it perfectly, I really want to know exactly how the futures will change. That's just not realistic, right? That's too specific. And then this would be less reliable decisions. So you want to learn some simple representation of how things evolve.

Like you would read in a medical guidelines book, right? It tells you if you give this treatment, the blood pressure will increase. You know, that's it. It does not describe all the other features. It really just describes what's necessary. So the AI explains to the humans how kind of things work given to data.

On the second point about robustness, uh, we can talk about where that uncertainty comes from, where that [00:31:00] instability comes from. In my mind, there are two. One is unaligned data source. Because, you know, data collection mechanism can be problematic bias for error prone. And then the second layer potentially is the model architecture layer, which I don't know as much as you do.

Um, so maybe you can elaborate a little bit more on that. Well, Ray, you nailed it. You can become a professor right away. So the first part you mentioned is exactly how we think about robustness. It's about the data. For example, let's say I collected data somehow, right? And then let's say, hypothetically, I could go back in time and reshuffle the randomness of the world and reconnect the data, I will actually have a different dataset.

And both these datasets, they represent the same reality, but these are just different samples. Just because of the randomness of my connection, I collected different datasets. Now, when I train a model on [00:32:00] dataset two, it will give me different things, actually. It's because the model is fully dependent on what data you give them.

If the randomness of my data sets affects a lot to my outputs, then that's not a reliable algorithm. It means, well, sometimes it would give me good stuff, but sometimes it would give me bad stuff. Like if Whole Foods uses machine learning across the US, on average it gives me a good performance, but the variance is so big, meaning in some stores it will be great, but in some stores it will be terrible.

And that's not what we want. Typically, we want algorithms that are more stable and reliable. Not only that, but we want them to give us guarantees. It tells us if you use this algorithm on data, then it would be at least as good as this and you would respect it. The part that you just talk about, it's more about handling the sampling variance, the uncertainty in sampling.

But what about the errors in data? Like how do you disturb data in a way [00:33:00] that to simulate that your data could be very off? That's an excellent question. Actually, second paper in my PhD was about this sampling randomness. as you called it. Now, sample randomness is always there. Whatever application you have, that's a fundamental thing in the data, but also was very often there is noise in the data, wrong data points.

There's a lot of problems that are, there's distribution shifts. Maybe the data you collected represents some population, but the way you're going to apply it is another population. We collected your data in Seattle, but you're going to apply it in, in, in LA. So there's a lot of different aspects of what we call corruption or perturbation in the data.

And that also is a main topic in robustness. And actually, ideally, you want to deal with these two, two at the same time, because both of them are most often present in the data. And that's actually what we try to study. Can you help me think through, [00:34:00] is there a difference between data drift or rather it's not data drift is situation changes, right?

Like then data shift. Um, How do you differentiate that? Like, is it random shift or this is some type of meaningful shift that the algorithm needs to adapt and evolve? That's, uh, that's a very important question and very much an active research area. One big challenge is this distribution shifts question in the sense that the data you're trained on does not necessarily represent what you're going to get in the future, but something similar to it.

And the big question is, What does something similar mean? How to model that? What does it mean? So there's really different scenarios. Let me give you a few examples. Let's say again, I trained in, uh, California and then I test in Massachusetts. You know, the populations are kind of different, [00:35:00] so there would be some sort of shift.

Let me give you now another one that is the same conceptual idea, but it's very different. Let's say now I train the classifier to recognize dogs versus cats. But I trained it on real images of dogs and cats. And now I'm going to test it on drawings of dogs and cats. There's some sort of shifts, like the data that I trained on is not exactly what I'm going to test on.

And how do we model that, actually? How do we? understand how to build algorithms that are robust to these shifts. What I'm going to say next is my own opinion. I don't want to claim that the community is saying something, but the feeling I got is that in the literature, many people want to develop universal algorithm.

I have a way to train my neural network that will generalize whatever the shift you have is in the dogs, cats setting and in the population setting and, and in the time series. And then, but I think that's, [00:36:00] doesn't really work. I think one has to understand each shift in the data, understand kind of the nature and develop algorithms that are adapted to the specific nature.

Like if the shift in my data is the pictures that change from real picture to drawing, it's conceptually not the same thing as the case of the population, for example. So you can develop a very general algorithms, but they're not going to perform very well because You know, these algorithms are too cautious.

They're like, any shifts can happen to me, so I need to protect against all that. But inherently, if you're too cautious, it's hard to have something very precise. So you need to have as tailored as possible. A second thing I want to mention about that is, there was actually a paper from someone at Meta, uh, not so long ago.

So this field, by the way, is called domain generalization. of domain adaptation, who published a review paper and he said, you know, there's many algorithms that claim to solve this problem. So what he wanted to do is to create a [00:37:00] uniform benchmark across all these papers, across all these algorithms, and to try all of them and compare them in a fair way.

Because in each paper, they would try the algorithm in some data. right? And then they say, Oh, it's worse, but these are different data. So you're going to standardize thing. And the fascinating thing, kind of funniest way that he felt none of the algorithm really improves over the naive classical approach, which is called the ERM.

So all of them It looks good because on their specific data sets, but when you drive them, nothing really works. But, and I think it's not surprising because I don't think it's possible to have a universally like robust algorithm to distribution shifts. So the second thing I wanted to mention is that in practice, I think what really works in this problem is actually just have more data.

That's somewhat the, kind of the conclusion. I don't have access to OpenAI secrets, but I don't think the algorithm they use to [00:38:00] train is that sophisticated, at least in the vanilla version. But what really works for them is that they have so much data, literally all the internet data. And I think that's really what works.

But that's not the end of the question, because in many applications, you cannot have all this data. So you need smarter algorithms to be able to, to cope with that. And it's something that I'm trying to think about now. The sweet spot in between is to think, okay, I don't have much data, but I have a little bit of budget to connect more.

Then with this limited budget, what's the right amount of data to collect? to deal with whatever issues that I have, shifts or Yeah, I think in practice, you definitely can allocate some budget into collecting the so called underlying truth. But the thing is, the world constantly changes. So first of all, have the budget to keep it running almost like indefinitely.[00:39:00] 

The second thing is, once you start intervening the world, the world changes, right? And so like, the so called underlying truth doesn't give you an alternative universe where you can just Go back in time and run the test and know what would have happened as we talk more and more, we get to a philosophical layer of like, do you really know what's really happening in reality?

And so far, my conclusion is we just have to accept the uncertainty that we're facing and to our best ability to measure how much potential uncertainty that there is. Yeah, I think you said here two very important things. The first one is, uh, this kind of interaction. When you make a decision, you also affect how the word becomes.

And it's something you can also try to estimate this kind of correlation between your decision and how the uncertainty changes. And the second thing to [00:40:00] have the reckoning that there is an uncertainty that is fundamental and we cannot completely overcome it, but we can quantify it and understand it.

and have decisions that acknowledge the existence of this uncertainty and are, you know, protected against it. But in the sense that we're not going to have the perfect decision, but we're going to have a decision that is aware that there is this uncertainty and that acts kind of accordingly. Yeah, for sure.

Wow. I have to follow up on the conversations. Uh, and we should really talk about potentially how to leverage your framework and thinking already in the field that I'm dealing with. Again, fraud detection, I think is a fascinating space. In school, you know, fraud detection is presented as a manila cut type of a classical ML.

AI use case. But when you actually get into the industry and face everyday challenge, you just realize there's a millions of new ones that in school you [00:41:00] never think about. And it's actually the nuance that really, um, matters. And I mentioned you have collaborated with the industry to implement some of this work.

Well, this is a tricky balance in academia. If you want to start with a practical idea, model it as a research problems and do the math, like very rigorous math of it and build a model. And then the model kind of creates an algorithm and then goes to industry, do the case study and then implement it and work with the people to make it work.

These are very, very long projects. And the problem I feel like is with the way academia is kind of structured right now. It does not allow you to do this whole path. So you need to focus on a part of it. For example, I worked so far in my PhD on the first part of it in developing the theory and the algorithms, right?

If I take my [00:42:00] algorithms and then I go to industry and then I work on implementing them, this is like two, three more years of research. The problem is that. The 10 year system, basically you have like six, seven years. Uh, and after that you have an evaluation and then they say the weather to give you a 10 year or not.

And you need to produce during these years. And if you focus on just one single project, which will take you five years, for example, you're It's a little bit risky, right? Yeah, this is fascinating because the end goal for you is to create a positive impact in this world, right? So if the research is isolated at the theoretical layer, it's hard to get to the part that you actually really want to do.

What you said is totally right. I've thought a lot about that impact part. So there's two ways to think about it. If you ask any theoretician, or actually most of them, The answer would be like theory has impact. It's just, it takes more time, right? [00:43:00] Like linear algebra and it's very much has a lot of impact today because like all machine learning is based on it.

Back in that time, it'd be people were like, who cares about this? Right. But today it's very impactful. You know, in, in operational research on machine learning, we're not developing theorems to the level of creating a field of linear algebra. Our theory would be closer to having an impact, me being there.

in, in, in years, but anyone who does theory at any level from pure math to like a wide to ML will say it has impact. It's just, it takes a little bit more time. So that's one, one way to think about it. But another part of me, and I think I'll be trying to do that within the next year, is I think I would still want to.

more of a little bit in the spectrum. And lately I have been invited by a big bank somewhere in Asia. Although, you know, finance is not the application I would have thought would have a positive impact, but you can try to have more. Uh, [00:44:00] applications of the theory. Okay, last part of the conversation that I kind of want to go into is you sort of already dropped a lot of hints around this, which is why did you decide to become a professor in the university?

A huge component is the research that gives you the freedom of studying the problem that you actually want to study. But also there is a teaching component, I guess, that sort of goes back to your experience of like wanting to reduce that information asymmetry. Is that the right interpretation for why you decided the professorship instead of going into the industry?

There's many reasons to it. I just said the research is one part. Honestly, the luxury of being able to work in whatever you want with an asterisk, because there are some constraints, you know, you need to have research that people would appreciate that gets [00:45:00] published and everything. So that kind of influenced it a little bit as well.

But having this luxury of thinking about the problem you want and working on the problem you want is to me really priceless. You're working for yourself. You decide every morning you wake up, you decide what you want to do. That's an immense luxury. That's one part, that's the research. The second part is the teaching, as you said.

I think the most rewarding part of my PhD was, somewhat unexpectedly, the teaching part. You know, research, it does have an impact or you do hope for it to have an impact, but it's more long term. But the teaching was immediate. Like you, if you give a good lecture to people and at the end, students come talk to you and they're like, Oh, wow, I never understood this, but now I do.

And you're like, Wow. I really see immediate impact and it's so, so how do you say gratifying, right? It's so [00:46:00] fulfilling to feel that. So being a professor gives me this opportunity to do something that I love and that helps people. And the third thing that made me want to do this is the university environment is to me really amazing.

Just a lot of young people around you, a lot of brilliant people, a lot of very smart people important problems. There's a lot of community kind of going on a lot of events. Sweet. Um, have you ever thought about your teaching principles? Like, what are the principles that you're going to operate by? You know, when you apply the academic applications for professorship positions, you have to write this thing called the teaching statements.

And they ask you, what's your teaching philosophy? And then you're like, wait, I never thought about that before. Now I kind of have to reflect. Honestly, I [00:47:00] don't know if I should say that. I mean, I hope the other professors don't, don't listen to this podcast. But, you know, when I was TA and I never thought about this, I just, went there and did it how I felt, you know, just kind of improvised, I guess.

But apparently you have to think about these things before. So I kind of learned that afterwards. I think the most important thing in my small experience in teaching was to be able to put yourself in the in place of the students. Try to put yourself in their bodies and, and mind and how they think, because obviously if you teach something, it's something you master really well and you've seen many, many times and seem easy to you.

The ability to understand that it's not like this for the and understand from their perspective what's tricky, I think is the most important skill that one has to develop. And it is [00:48:00] not trivial at all, actually, because once you see something, it's hard to unsee it. It's hard to put yourself back in the position where you could not understand it.

And even more than that, because your background is, I don't know, math or whatever, It was easier for you to study it than someone from a different background. So that's also something you need to try to factor in the way you see the problem. So being able to see from the student's eyes, I think, is the most important thing.

It allows you also to be more caring toward the students, because then you see them struggling, you understand why they're struggling, you understand why it's hard from their perspective. And it allows you to explain it to them. In a way that from their perspective, it would be easier to understand. I think that's really what helps me.

A lot. I think I probably should adopt a similar principle. Fundamentally, a lot of problem is communication problem. And the communication problem comes from upon some self [00:49:00] reflection is my inability to understand where the other person is at. And in the initial phases, my immediate reaction is, how could you possibly not understand that?

Right? And I guess for me, It takes a lot of friction, uh, to accept that I really need to put into the work of understanding where the other person is at. Another thing I think is try to make the person feel comfortable. Yeah. Because a lot of people, they don't trust themselves. They don't believe in themselves.

They're like, I can't get this just because I'm not good enough, but trying to make them feel comfortable and get them to. self cultured and spike, I think helps a lot as well to tell them, you know, it's normal, it's okay. And then when they feel like you give them this confidence, like, you know what? Yeah, it's true.

I can actually do that. And that helps a lot. Thank you for [00:50:00] the tip. I will try to practice it. All right. I think that's it. All the questions that I have so far, I really enjoyed this entire process. I enjoyed it too. Thanks for having me. That was, that's amazing. I'm looking forward to actually listen to it myself.

This could be the last episode of Floating Questions, or it may not be. Either way, I hope you enjoyed flowing along with us today. If you liked our journey, please consider subscribing. Thank you for listening, and may the questions always be with you.