"Biology's Image Detective" - Fraud In Science

Elisabeth Bik, PhD, is a science integrity consultant who specializes in finding image duplications in scientific papers. After receiving her PhD in Microbiology at Utrecht University in The Netherlands, she worked 15 years at the Stanford University School of Medicine on the microbiomes of humans and marine mammals. From 2016-2019, she worked at two microbiome startup companies. In March 2019, she left her job to become a science integrity volunteer and occasional consultant. She can often be found discussing science papers on Twitter at @MicrobiomDigest, writing for her blog ScienceIntegrityDigest or searching the biomedical literature for inappropriately duplicated or manipulated photographic images and plagiarized text. She has reported over 4,000 papers for issues with image duplication or other concerns. Her work has been featured in Nature, Science, Wall Street Journal, New York Times, Washington Post, Le Monde, and The Times (UK). In April 2021 she was awarded the Peter Wildy Prize by the UK Microbiology Society for her contributions in science communication.

Twitter: @MicrobiomDigest

New Yorker Article - https://www.newyorker.com/science/elements/how-a-sharp-eyed-scientist-became-biologys-image-detective

Favorite Study - "Host lifestyle affects human microbiota on daily timescales" by Lawrence David - https://genomebiology.biomedcentral.com/track/pdf/10.1186/gb-2014-15-7-r89.pdf

If you would like to support Dr. Bik's work, consider becoming a patron at: https://www.patreon.com/elisabethbik

Timestamps:
00:51 - Biology's image detective
01:59 - The scope of fraud in science
03:37 - What is motivating scientists to commit fraud?
04:46 - Dr. Bik's early work with plagiarism 
08:54 - Incentive structures that lead to fraud in science
12:09 - Peer review and the role of editors at science journals/newspapers
14:44 - How can we change the system?
16:07 - How can non-PhD's approach reading scientific papers?
18:27 - Are pre-publication sites good for science?
21:14 - Hydroxychloroquine
24:49 - Ongoing cases against Dr. Bik
30:12 - Public perception of scientific integrity
35:24 - Algorithmic fraud detector
38:43 - Dr. Bik's ultimate mission 
40:40 - Dr. Bik's favorite paper

Hosts:
Vasanth Thiruvadi - @NextVasanth
Faraz Abidi - @fzfromcupertino

Intro Music: "Pain is the Essence" remix by @AdiSoundsGood on Twitter 

Transcript

0:00

I just feel there has been so much damage done to the credibility of scientists. You know, in, particularly in the past four years under a president who did not seem to support science or you know, even suppressed, good information. And also the, the power of social media, where there seems to be very little incentive by the social media platforms to. Bring out only the truth and to suppress false information. And I can sort of see that we want in the United States in particular, we want freedom of press and freedom of opinion, but I also feel that there should be some way of reporting people who. Who make false claims, like the vaccines have killed more people than COVID-19 itself. So well thank you for coming on the podcast to listen. Just to get started. I wanted to. Kind of ask you. The new Yorker just published an article about you and in the headline that described you as biology's image detective, can you explain what that means? Yeah, so, well, I'm a biologist by training. I'm a microbiologist and I also apparently have some talents to find, or to see duplicated parts of images or duplicated images. So I kind of detect a Photoshop, but I can detect if a photo has elements that are repetitive. So for When we talk about scientific images, that could be an image showing multiple cells, but all the cells look identical. So it appears that those have been cloned by Photoshop. And so not about biology cloning, but by somebody taking a cell and stamping that a couple of times in a photo. And so that is my specialty. I look at photos in scientific papers and I will detect duplicated elements or just panels that have been tested. How big of a problem is this well, that is a big problem because that means that somebody has. Manipulated the results and photos in scientific papers are the data. When you read a scientific paper, it will say we found such. And so, you know, this, this isn't that experiment and this was the outcome. See, figure one. So figure one, any figure in a scientific paper usually is data that's. It's a photo of cells. It's a photo of tissues. Of course also be a line graph or a table or things like that, but yeah, figures are the data. And so if somebody changed the results in a photo, or if somebody used the same photo to represent two different experiments, Might be science misconduct, that's data, fabrication or falsification. And that's, that's a really big, no, no. In science you should not be doing that. Is this a widespread issue? No, it's not. It's it. When you look at my work, you might think it's a widespread issue because this is what I do, and this is what I find. But I actually did a. Research to look at how many times I would find these duplicated images. And so I scanned 20,000 papers that had at least one photographic image and 4% of those papers had a duplicated image or duplicated parts within an image. So it's about 4% and the real percentage of fraud might be higher because I'm only looking at photos. And so I'm not looking. Tables or sequencing data or any other type of data. So the real percentage of fraud might be higher. I would just estimate between five and 10%, but I'm only probably detecting the tip of the iceberg if you are a really good Photoshop or I wouldn't find that. But yeah, so it's 4% of detectable duplications in biomedical papers. What's kind of the scientist's motivations here. They're risking their careers, right. And their professional reputations by conduct conducting scientific misconduct. And more importantly, there. Putting out false, false science and putting a bad science. So why are they doing that? Because the repercussions are surprisingly small, like a lot of scientists who do this and who are being caught, don't get any punishment for that. Like they add best some scientists. And this is really. Only a very small fraction of scientists who have been caught. They might maybe be punished by not receiving grants for a year or something like that. But that's for most scientists who already have a lot of brands going on, probably not a big problem. So a lot of scientists who are being caught doing this are still yeah. Still having an at, at a university and I'm not being fired. That's frustrating because it's cheating and we would, you would think that the person would be punished for that, but there's very, very little punishment. And in most cases, people are, yes. They'll have a glorious career and get more and more. Right. When did, when did this first come across your radar as something you were interested in and an issue that you saw was significant? I started his work working on plagiarism, actually. So I heard on a podcast or I was reading about science. Misconducts, but specifically about plagiarism. And I thought let's just check a random sentence that I had written in a science paper in in Google scholar. So I put it, put that sentence between quotes in Google scholar expecting only to find my own paper. And I found another paper published. Predatory publishing book or like some, yeah, some, some strange online book that was free for download, but it was actually my texts. So they had used my sentence and pass it off as their own in this paper. And it turned out that this paper, not only I'd use my sentence, but the sentences of many other scientists. So it was sort of this, this patchwork of many different scientists at different, many different papers that they. Put together and sort of passed off as a new paper, but it was all plagiarized text. So I, I worked on plagiarism for about a year. And then by another coincidence I spotted in a PhD thesis, a Western blots. It's a protein bar lot. It's a photo. And I, it had a very particular smear that are recognized. And then I saw a couple of pages in another chapter or so in that sense, PhD thesis. I spotted the same photo, but it was upside down and it had been used to represent the different experiments. And, but yeah, I recognize it. It had this weird little spot or smear, so that was not good. And this paper had been published in a scientific paper as well. And I recognized I had some talent to, to do that. So it was all by coincidence mainly, but it. It's one of those moments that sort of make your career or make your career change? Yeah, if I hadn't seen that, then I probably would never have worked on this while you were doing this. What was your day job? I worked at Stanford, so I was a microbiologist. I worked on the microbiome of Marine mammals and humans. So the microbiome. The bacteria that live inside our bodies and, or on our bodies on our skin. And I was working on, on the microbiome of humans, but also dolphins and sea lions. And that was my day job. I was, I guess, a regular scientist working at Stanford and writing papers, doing research. And I was doing this image duplication searches in the evenings or in the weekend. So it was sort of my, my hobby. I yeah, I saw, well, I think you're, you're being a little humble here. Cause I saw on Google scholar that you have a paper that has like 20 K citations, something crazy like that. And I, I worked in a PhD lab in college and so I knew a bunch of PhD students here and PhD is you're by far the most cited person. I know there's, there's plenty of other people who will play. I'm just a very modest scientist by their standards. So there's always, people have published more, I think you know, four, four at the point. Of my career that I'm at, I am a probably, yeah, sort of a middle of the pack type of scientist, but yes, there's one paper that we published in science and I'm the second author of that. And that paper has a lot of a lot of citations is what was one of the first publications. Analyzing the microbiome of humans using DNA sequencing. So there had been other papers. We were not the first, but it was one of the first large-scale papers. And that has been cited a crazy amount of times, I guess it was published in science. And so I was incredibly lucky to have worked on, on that project. And yeah, I think the, the paper still stands as we, we, we made sure it was high quality and. No image duplication. Oh, wow. There's actually no phone auto in it. So like most scientific papers. There's actually no photo in this. Just, just line graphs. And but yeah, I can vouch for it that there's no, there's no science misconduct. I'm sure there's errors in it. Like in any paper that, in which you analyze, you know, thousands and thousands of, of DNA sequences, there's. It's very hard to not make any errors. We all make errors. But it's it's done with the best of intentions and it has stood up to the, to the test of time. It's still being cited. So Elizabeth in do some research for this pod, or this particular subject, I realized that there are a number of ways to publish fraudulent data in science. The one that you specialize in, which is image doctoring it, I guess it's somewhat prolific, like what you just described. But then there are other ways like people will run experiments. Nine out of 10 times the result is no, but then like one out of 10 times, it ends up being exactly what they want and then they get that published. So what's your sort of take on the full wide variety of flawed in science and whether the incentive structures that allow image doctoring to happen or the same instead of structures that allow this to happen. How would you break down sort of the incentives that, that lead to both? Well, so the incentives in science are, are to publish. So we, as scientists are. Encouraged, but also almost forced to publish because it's needed for our careers, like as a postdoc or a professor you need to, or you're expected to publish an X amount of papers per year. And unfortunately, scientific publishing focuses on positive results. So if you have done a lung study showing that a particular drug does not help against the particular cancer, that's not a very publishable. Paper, because it's sort of a negative result. It shouldn't be, but unfortunately, a lot of journalists will say, well, that's result is not very novel or, you know, earth shattering. We want to have a positive results. So the incentive to publish positive results is one of them. Important. Yeah. Incentives to cheat because people want to have a positive results. And like you said, if you have you know, 10% only 10% of your experiments gives the results you would like, then that's the experiment that you'll pick for your paper. So that's called cherry picking is basically picking the results that you like to see that fits your own hypothesis. But ignoring the results. They do not fit your hypothesis. And that would be also called publication bias or like we are, we're all biased. We all want our experiments to work out a certain way. And if it doesn't do we accept those results or do we keep on trying until we have a positive results? And, and that's still a big step to where science misconduct. I do feel that it's, it's cheating in a way, but it's. I feel as bad as really faking or forging results. Like when you have have you met, if you have measured a couple of things and you change the results, you've changed the values so that they cross a particular threshold and suddenly your negative sample becomes a positive. That is where we're really talking about science misconduct. So there's, there's a whole range. Steps in between from publication bias, towards P hacking, which is sort of the cherry picking where you keep on doing statistical tests. And there's one statistical test in which your results are only is significant. You pick that. Really changing or fabricating results that which so fabricating and falsification, those are considered science misconduct to get to with plagiarism, by the definition of the office of research integrity, she hacking and publication bias are not necessarily included in the pure definition of science Smiths for that, but there's a lot of gray in between. There's, there's a, it's hard to draw the line. What is misconduct and what is. Bias. Right. I think in reading the New York article specifically, I was kind of surprised to find that you had found issues and some of the most important and prestigious journals and articles. And when I'm always listening to news or other podcasts about science, they're always referring to peer reviewed articles. Or these editors at these journals have the highest standard. But obviously not because, because you've found you've, you've discovered some sort of fraud. What are these editors missing? Like what is, what is the issue within this organization that allows stuff like this to get published? Oh, there could be all kinds of issues that editors are either not paying attention to, or just not trained to find problems with papers. So you would hope that an editor would find them. Yeah. Obvious Photoshops in, in, or most obvious errors in papers, but an editor is usually a person who's unpaid who does editing sort of as a side job, but might be a very busy professor who being asked to do an editing. B being editor of a journal. So basically what they do is they, they got the manuscripts that are usually pre-screened and then did need to find peer reviewers, or also busy and unpaid and, and have no really no time to really look carefully at a paper. And and then when they received the peer reviews, they sort of compile that and make a final decision. But very often I've been an editor myself for a very short amount of time. I found it very hard. It's You don't really have time to read the paper yourself. You sort of rely on your peer reviewers to do a good job. And sometimes they also do don't do a good job. So it's, it's really tough because none of these jobs are paid. And unfortunately we have to pay the publisher a lot of money to get our work either depending on the model of publishing to, to get the paper published. So where that money goes into is, is not very clear. Everything is published online. So it's. And, and also editors are not trained to find these problems. I'm looking at it with a lot of experience. I I've seen a lot of different types of Photoshops or photo duplications. And so I'm very trained for these situations because I've seen so many of them, but a lot of people don't really see these problems until you pointed out to them. I point point I do a lot of puzzles on Twitter. I will post them on their image forensics. And then of course, when I write. One of those images, people know there's something wrong and they'll usually find it. But people forget that I've seen hundreds of images maybe before this one with the duplication. And so once, you know, there's an arrow, you'll find it. But if you just quickly look at fairs, you might not spot it. And you need to be told that this might be a thing before you start to see that yourself. So if you were a benevolent dictator, what would you change about the system to avoid these issues? It's really hard because fraudsters are going to fraud. And the only thing you can do is, is to like ask people to ascend in raw data. That would be sort of an extra hurdle, but if a real fraudster wants to fraud, they're going to fraud. So I, I have no illusions that. Make science a hundred percent foolproof and fraud proof. There's always going to be people who want to cheat the system because we, we put so much emphasis on outputs on science papers as, as the output of scientists. And it's only when we have replicated a paper that we sort of know it was probably true, but you can never a hundred percent be certain that everything in the paper was honest and that's very unfortunate. Science in a way is, is about finding the truth. I've always felt like when you are in science, you want to discover a particular pathway or a particular bacteria, or you want to discover what is true. What is, what is the truth about a particular biology process? And so I've always felt that science should be about reporting the truth. So for scientists to fraud, I feel that's a huge violation of, of our profession as well. Yeah, I definitely agree with that. So I'm, I'm someone who reads a lot of papers. Anytime. I'm curious about something I'll hop on Google scholar and I'll, I'll see what I can find. But I I'm, I'm not a PhD. I don't have that much training in this field. And I have a lot of friends who do the same thing as me and don't have any experience with this. how do we read papers and say, this is something which we should, we should have high degree of credibility. And this is a really good paper versus this is something that, you know, maybe we shouldn't put too much confidence in. Yeah, that's a good question. I don't have a standard answer for you because even papers that have been published in high impact journals by. You know, officers who work at the institutions that seem to have some credibility, even those papers have been caught with fraudulent data. So it's not a hundred percent guarantee, but having said that papers that have been published in science or the Lancet, usually with some very big exceptions, usually are more credible than papers in that are, for example, published on a preprint server that have not been peer reviewed. Those are the two extremes, but yeah. There, there has been a big paper in published in the lenses that had been busted retracted last year, because it was based on probably based on fraudulent data. And so that's one of those big exceptions that makes headlines and that make a lot of people who are not scientists, think that all science is flawed. Well, that was really the exception. It's it's like saying, yeah, thoroughness, you know, it was a company that did not really do well, so we kind of trust any biotech company. Anyway, you cannot make those, those extra population extrapolations based on one bad apple. It's it's usually the. Those cases make headlines and for good reasons. And that was a fraudulent paper by at least from old evidence I've seen, but it was hard to recognize it as a fraught Olympic for, I did not recognize if myself, either. I actually tweeted about this paper, my haters, my trolls who are my loyal enemies on Twitter are still saying, oh, big tweeted about this paper. So she cannot detect any fraud. Right. And it was hard just to look at that paper and realize it wasn't fraud. You really had to dive deep into the paper, knew a lot about particular numbers that were misreported to find out that that was fraud. So it happens anywhere. Those cases make big headlines, but in the end, Usually you can trust those, those journals. But yeah, it's a, there are exceptions, of course. So, so let me, let me play a sort of devil's advocate really quickly, or at least from what I've read and what I understand by be totally off. I came across these pre-publication sites, right? Like I haven't written down here AR XIV and bio R X archive archive, bio archive, archive. That's how you pronounce it. Okay. Sorry. This is So not everybody knows where I have that on this argue. Right. And the argument that I basically read is, well, sometimes it's worthwhile to publish some sort of science output. Just get the output out there, even if it's just an idea, even if it isn't peer reviewed, even if it isn't a hundred percent accurate and has veracity. Just to get that idea out there, you know, into the, into the minds of people that might do more research and build on it, even though it's like incredibly low barrier to entry and anybody can get it out there. Is that generally a good thing for science? Do you think? I believe so. And especially in the case where we were last year at the beginning of an epidemic where. Quite frankly, we're all in a state of panic where that was, you know, a lot of mortality, a lot of people dying, a lot of people getting sick, a new virus, nobody really knew about, you know, th the new enemy was in, in a situation like that. We need science to be fast, and we need to have a very quick model of scientific publishing. So if a person has found a result that is worth sharing, that might save lives. There's a big argument to make, to publish this quickly, even though it might mean publishing before peer review, but just getting it out there so that a lot of people can read it and, and benefit from these results. But there's a delicate balance between wanting to publish fast and doing good science. So those things are. Yeah, they're, they're, they're two ends of the spectrum. It's, it's two parts that are usually not in agreement with each other because science, if it's done well, it's very slow. It's a Spain awfully slow. It's like looking for tiny details, having long arguments with other scientists about how to interpret the particular results. Yeah, that just is not, cannot be done in a very fast way. And so it, it's, it's finding this balance between publishing results really fast, but knowing it's on a preprint server, it's not being peer reviewed. It's just a view of one particular lab. And that couldn't be very right biased because no other people have had a chance yet to carefully digest it and give feedback and go through these normal and slow processes. So it's. It's I'm all for pre-print service, but it comes with a lot of caveats. You need to interpret it as just a fuse of one lab, not being peer reviewed and take it with a grain. So one high profile instance that I think most people are familiar with of of a paper being just rushed out before it was ready. Was the, was Trump's favorite? Hydro, hydro chlorine. Sorry. How do you say that? Hydroxy chloroquine study. That's just like God got shuttled out and I believe you were one of the early scientists to say, this is, this is bad research. Right? So can you, can you kind of talk about that situation? Sure. So this was a paper by the group of professor Howell in Maaseiah in France. And he claimed that hydroxy cork Quinn was a really good medication to get rid of the virus. So he looked at patients who had the virus who were positive for the PCR, and he looked at clearance of the virus. Repeatedly testing these patients and seeing when they would become negative. And he showed in his paper, which was only, I believe 40 patients. So it's a very small study and he had three different treatment groups. So some people were not treated. Some people only got hydroxy Clarke when and the third group got hydroxychloroquine. Plus I see assay throw mycin, which is an antibiotic. And he showed that the the both groups that had the hydroxy Clarke. Treatment that those people cleared the virus. So got PCR negative faster than the people who did not receive any of those drugs, but the, the groups were really small. So if you have 40 patients and you divide them over three groups, you can already see that the numbers get pretty small, but there are a lot of flaws with this study. So one of the things was that. There were six patients who were in the hydroxychloroquine groups in either one of these groups who didn't most of these patients were, did not really do very well on the hydroxychloroquine and they were left out of the study. So they started with a particular patient group that's but six patients were left out. So one of them died. Three, I think two of them got really bad to got really sick. So they were transferred to the intensive care. One patient got little. Side effects are two patients and one patient just walked out of the study. So it wasn't pretty clear. So it looked like the researchers might have decided to do this cherry picking that we talked about previously, where, you know, the results were really quite what they had hoped. You know, if one patient dies on your drug, You should not leave them out of the study. Right? science, it's us. So basic, like just say, I did not want this result. I'm just going to leave it out. And that was actually, it had noted that they had written it in the paper. So who else might to have left? Left? That was just one, there were many other problems with this paper. So I wrote a long review about it. There was problems with ethical, ethical, the dates of approval, or first the start of the study that there was some problems there. He included some children, even though he wrote that he didn't think Lew children, but he did. And then. There were differences between the treatment group. So those people were all treated at a different hospital than most people who were not treated. And so there were all these differences between the different treatment groups that you would not expect. Red or rigorously set up scientific study, you would do shoots, randomize your patients. And he didn't do that. He appeared to have handpicked patients and maybe those patients who were on hydroxychloroquine were already less sick to start with, or maybe farther in their in their disease status. So they would have cleared the virus even faster. So all kinds of problems. And I, I wrote a, a critical blog post about that, and that got me into trouble. Can you elaborate on that? Yeah. So obviously professor Al did not enjoy my critique and and I can understand that I understand that he was not happy with my critique and I, so I raised the concerns not only by writing about this on a blog post, but I also posted on a website called pub peer, which is a website where you can leave comments on scientific papers. And he did not answer to my command. Stare in study, started calling me all these names. So he's called me a witch hunter can sing play, which means like a crazy woman. He called me a girl who hunted me down and, and no, all kinds of, not very nice words, but yeah. So, and he also some of his people who works for him who are working for him in the same institution, Started to harass me on Twitter. And so there was one one of these professors like shabby and started to ask me all these questions on Twitter. Which may boil down to who are who's paying you, are you being paid by big pharma to bring down my professor, professor And in the meantime, I started to look at more papers by that. Well, and found more problems. So there were some, actually some, some image problems. So some image, duplication problems, but also other problems with ethical approval of some of his studies. So it appeared that there's not just this hydroxychloroquine paper that had some. But also a bunch of other papers. So I posted all of these on puffier and other people started chiming in finding more and more problems. So by now I think this professor has 270 papers from his group that are, have been flagged on pop here. And he's becoming a bit annoyed with all of us and yeah, he has now. Threaten me with a lawsuit. So he has filed a complaint against me with the prosecutor in Maseo claiming that I harassed him and that I extorted him and blackmailed him. And that's all based on two answers. I gave them on Twitter where they asked me, who's paying you. And I said, oh yeah, You can donate money, I'll make patron accounts. And another one I said, well, I can, I'm a consultant. And so I could check papers if you want, if you want me to check some papers, happy to do so, as long as you pay me. So he claims that's blackmailing. I kind of imagined that's blackmailing, but yeah, it's you filed a police report with the prosecutor in the Messiah and this case is under investigation. And I, I hope this will not lead to a lawsuit because I don't think I did anything wrong, but I'm not quite sure how the legal system and friends work. So, so for now it seems to be. Threats to try to silence me, but I've already sat on Twitter a couple of times. I'm not going to be silenced. I'll keep on a stand behind all my questions. You can answer them on pop here. And I don't think that a scientist should be resorting to legal steps to silence your critic hustlers. But yeah, I guess that's. No, a couple of other authors in some cases have, but not all himself and should have. Yeah, I have not answered any of the questions. I saw a petition. I think it was like a thousand scientists who came out to support you. Why? I mean, how could you not, this is, I don't know, this, this is just so outrageous is unbelievable that like, not only is this guy putting out garbage science that has affected us that has affected the United States because the president used that as, as policy. But he's also going after. Whistleblowers who are trying to keep science clean. This is so outrageous. It is. Yeah. And unfortunately, that's, I'm not the only person who has been harassed or threatened on Twitter or even in real life. I have not been to that country in real life, but there are a couple of scientists who are just trying to bring out good news or, well, let's say honest news and try to. Go against people who spread misinformation. There's so much misinformation right now on social media, where, for example, there's, there's these tweets where people claimed that more people have died of the COVID vaccine than of COVID itself. And as a scientist, you kind of be silenced. You kind of look at these numbers and just looked out away because it's completely not true. And so a lot of scientists will say, that's not true. We'll get to here, here at a number. But then there's all these people are not scientists, usually who claim they know better. And they have fun to another website that disagrees with all these hundreds of scientists. And so you get all these very polarized situations where scientists try to bring out honest information and based on facts and other people say that the facts are. Yeah, all these, these things, these wars going on on Twitter, and sometimes scientists have been threatened. And there's actually one scientist in Belgium who is now living under police protection in a secret location because he's being threatened. Then I was the soldier who has escaped from the army or something with a little of weapons and it was trying to kill the scientist. And it's just very strange situations. We'll see that as well. Yeah, I know there's just this like rising tide of scientific misinformation. And I can think of one big reason why. But of course, social media has also, I think I contributed to this now. Just anyone can publish without any sort of. I dunno. I dunno, peer review is not the right term for when people who are not scientists say stuff, it's like the idea of sharing an article without reading it first. Something like that. Elizabeth, I think on this topic. And this is something I've been thinking about a lot, is that it just feels like in the since the pandemic, I guess it's not exactly what we're, but since it we're coming to an end to it it seems like the public. Really doesn't trust the scientific community or it's at an all time low. What do you think can be done to improve that trust whether it's in the United States or around? Oh, that's a great question. I. I don't really know to answer because I, I just feel there has been so much damage done to the credibility of scientists. You know, in, particularly in the past four years under a president who did not seem to support science or you know, even suppressed, good information. And also the, the power of social media, where there seems to be very little incentive by the social media platforms to. Bring out only the truth and to suppress false information. And I can sort of see that we want in the United States in particular, we want freedom of press and freedom of opinion, but I also feel that there should be some way of reporting people who. Who make false claims, like the vaccines have killed more people than, than COVID-19 itself. And, but yeah. Then other people will say, well, if you suppress that opinion, then that's oppressing freedom of speech. And I, yeah, I, I think that's a very hard to solve. Issue. And I, I sort of want this country to be about freedom of speech, but when that turns into misinformation, that could actually cost life. I do feel there needs to be drawn a line somewhere. And as scientists, we are all very frustrated that there's no way to report on Twitter, false information. There is actually no bottom. There's no way to report people who send me emails saying you belong in jail, or you are a fraud. Like I cannot report that I've reported several of these tweets and I always get to hear from Twitter. We don't feel that violates our rules. Like you can actually say a lot of things to each other before, before tweets are being taken down. And I, yeah, it's, it's this delicate balance between freedom of speech. Yeah. Still trying to be polite to each other, and I'm not sure how to solve this, this this is a very important question with with a lot of aspects to it and yeah, just don't have the answer. Do you do you think all scientists, whether it's physics, math, biology, geology, whatever have a problem with false data or is it just a bigger issue within certain subsets yeah, so I feel fraud in science is probably anywhere in any particular field of science. I focus on images, which are a part of molecular biology type of papers because they have a lot of protein blots or DNA blots. And so those are generally photos, but there's also. A lot of other types of data, like optical spectra, where I found fraud in I haven't really looked into a lot of other fields, but I do feel there's probably fraud everywhere, but I don't know enough for example, to, to look at the mass paper or a geology paper to find a potential problem in it, because that's not really. My background, I, I look at these papers and I just see numbers or graphs, and I just don't understand what they mean. So I kind of detect problems in them, but I'm pretty sure that fraud is everywhere, but I also think it is important to be it's, it's easy to listen to my story and the story of misinformation and scientists and, and I want to make sure that we. Confuse these two things because there's fraud in science and that's what I work on, but I also want to make sure that there's, that most science is to be to be trusted. And I feel it's very easy to hear my steel, sorry. And interpret like, oh, all science is, is flawed. And we kind of trust that. And at the same time I'm telling no, we should trust science. And I, I feel that's a very important thing to To distinguish between these two things. So I, I will say that there is fraud in science. It's probably everywhere. There's fraud everywhere. There's fraud in banking, in construction. You know, what is there, there's probably no field that you can think of that has no fraud. So science is not immune to that either, but as a whole, science is about finding that truth. And and it's the only solution we have. I feel to solve the big problems that we're currently facing in the world. Epidemics and climate change and, and things like that. And I think by now, most people will be convinced that for example, the earth is not flat. And I feel that a lot of these misinformations in science are based on, you know, the earth is flat. Data, like there's, there's no real data to believe that that's the case, but people, if they want to believe that they'll believe in that. Right? The reason we ha the reason we live longer than 40 years and we have cell phones and the internet is because of science. Exactly. I, I yeah, I I'm, I'm like, at this point I'm like, we need more Elizabeths in the science community, but people like me, I'm not, I'm really not the only person, but the most of these people work anonymously for good reasons, as you can see, because I'm being haunted down by the French you know, disinformation, trolls so most people will choose to, to do this work anonymously, but I'm definitely not the only person doing this topic. Do you think that in the near future, we'll be able to train an algorithm to spot the doctorate image if given like 10,000 images, basically give your eye your particular unique talent to an algorithm. Is that a possibility? Yes. And I actually, I'm going to take a sip of water because I turned this into a drinking game because I get this question so many times on Twitter. So I'm just going to take a sip of water. Delicious. So a lot of people will say, oh, I can, I can ride it tool on a Friday afternoon. That can do what you can do. It's much harder than you than you might think, because a lot of these duplications are not pixel to pixel identity. So science images have usually been compressed a lot. They're inserted in a PDF. There's all kinds of image compression and data processing that. Made one image looked like an outer, but not image, not pixel to pixel. So you're kind of. Do your standard pixel to pixel comparison and find these things. A lot of them, a lot of the times the images also rotate or zoom dinners ripped out or like mirror. So it's a little bit more complex than that. And I've actually participated in a DARPA challenge where I came with my data stack of flawed images and good images. And nobody could crack the code. There were several groups that, that all claim that they could ride on a Friday afternoon. They could write this probe into detectives. And we're now three years later and now they're starting to develop tools that can actually do this. So it's, it's pretty hard. But on the other hand, yes, this is information. This technology will be there and it's, it's actually getting ready. There's a couple of tools I'm already starting to use that are. Starting to find applications, but, and in some cases they're better than what I can find. They're definitely faster, but there's also duplications that I just see what my, just my eyes and the suffer just cannot see it. I'm like, come on. This is there. It's so clear. Yeah, so it's, we're still, we still have a long way to go and it always needs human interpretations. Software can easily, maybe in the end pick up a duplicated image, but in some cases there are duplications that are expected and actually quite normal. Where, for example, you do a control experiments and you compare to two particular drug treatment, but then later you have the same photo of the control experiment, and you compare to another experiment. So in those cases you might see the same photo. But it's a total, a totally normal and acceptable way of, of reusing the photo. So the software might still detect it as a duplicate, but then you need a human to interpret. Yeah. Well, this is actually the same experiment, so that's fine. The advantage of software will be in the end that any image in a manuscript that is sent into a journal could be scanned against the database of all images that have ever been published, which is, you know, competition was still challenging. Something that I expect to be solvable so that people who want to reuse an image from an outer group from an older paper will be caught. And that is something I could never do. I can only compare a couple of images to each other, but. I cannot remember enough of them to remember an image I've seen three years ago. I would not remember that. The image. Sure. So yeah, it sounds like the software will play a role, but it will never be able to totally replace that human factor. I think it's fair to say that you've had like an undeniable impact on science. But as you look out into the future, what is sort of your longterm aspiration impact that you want to have? You know, what's the north star of what you're trying to work towards. I hope there will be more well punishments is always a big board, but like, like some way that people who are caught doing fraud, that there will be repercussions for them because I feel there's, there's too many cases. I've reported to journals in institutions where our data was just simply no reply. So about 60% of the papers I've reported in the past years have not been acted upon. These are papers with very flawed images. Some of them just simple errors that could be addressed with a correction. Some of them like re outrageous Photoshop jobs that are so clear to me in five seconds, that there is something that is very fishy going on there. But five years down the road, these papers are still out there. And so. I'm looking forward to work together more with journals, with institutions, with publishers to very quickly address these, these problems and not have them look the other way. There are so many conflicts of interests where journals do not want to respond because they might lose their citations. They might lose their image. As a, as a, you know, a good journal that would never publish any fraud and institutions also do not seem to want to address these cases because maybe they have a very famous professor who is being accused of something bad, but he or she brings in lots of money. And so let's just pretend this didn't happen. And so I'm looking forward to. A time where these cases are swiftly addressed and where there's much more room to give money to honor scientists and not the scientists who cheat and that's still a long time. Cool. So yeah, we, we have a bunch of admiration for your work. I'm sure all of our listeners will too. I just want to wrap things up now by asking you, what is your favorite. Oh, my gosh. One of my favorite papers is that of Lawrence David in which he. Sampled himself sampled his microbiome. So took his own stool samples and followed himself and another scientist for a, about a year. I looked at how his, the composition of his bacteria in his stool changed over time. When, when for example, he went camping or he went like people sick or went to another country and you can see it. You can see the stability of the human microbiome. And you can see also the periods where the microbiome just changes because he got sick or, you know, the little things we go through over time. And I felt that paper was so important to show the enormous stability of our microbiome. And which is amazing because we eat different things every day. And so we feed our microbes different, different foods every day, but it's pretty resilient to the changes that we, we we bring along to it. But when we have a big change, when we got really sick or we go to a different country, this is where the microbiome of the human changes. And I thought it was so elegantly done. That was one of my favorites. That's I would love to read that it, could you send us a link and you can put it there? Of course. Yes. It's it's an all okay. For, I haven't really kept up to date with microbiome papers, but so it's probably all my questions around eight years old or so, but yeah, it's a, it's I'd love the paper. Just has some really cool graphs. Oh, nice. Yeah. And it sounds like such an interesting story that he's telling you to know. It's not just one piece of research, but this, this guy's life that is being explored through science. I think it's a very poetic yes. All right. Well, thank you so much for coming on the podcast. Elizabeth has been such a cool episode. One of my favorites so far, and I've learned a ton. Well, thank you for us. Thank you. For sense. Was my, my pleasure being here. Thank you. Thank you so much, Elizabeth. I really appreciate it.