Deepfakes and Synthetic Identities: The Erosion of Trust in Online Content

In this episode of Integrity Insights, Filip Brokes speaks with Emmanuelle Saliba, Chief Investigative Officer at Get Real, about deepfakes, synthetic identities and AI-driven identity fraud. They discuss how generative AI is changing misinformation, social media manipulation, voice cloning and remote identity verification, with a focus on what these risks mean for businesses, governments and individuals. The episode is relevant for listeners working on corporate security, financial crime, fraud prevention, cybersecurity, OSINT and digital investigations.

Topics covered

Deepfakes and synthetic media
Visual verification and investigative journalism
How online misinformation has evolved
Why simple deepfake “tells” are no longer reliable
AI-generated influencers and synthetic personas
Commercial scams and engagement farming
Political influence operations and synthetic social media accounts
Voice cloning and real-time impersonation
Deepfakes in war and crisis situations
Provenance, watermarking and verification standards
Corporate exposure to AI-driven identity threats
Remote identity verification and synthetic IDs
Deepfake-resistant security systems
The threat horizon for businesses, governments and individuals

Emmanuelle Saliba: https://www.linkedin.com/in/emmanuellesaliba/ 

GetReal: https://www.getrealsecurity.com/ 

GetReal Youtube: https://www.youtube.com/@GetRealSecurity 

Watch The Whole Podcast On Youtube: https://www.youtube.com/watch?v=St4FL0OmKjk&t=260s

Connect with Us:

• LinkedIn: https://www.linkedin.com/showcase/integrity-insights/?viewAsMember=true
• Berlin Risk Linkedin: https://www.linkedin.com/company/berlinrisk/?viewAsMember=true
• Website: https://berlinrisk.com/

Transcript

 Hi, Emmanuel. Welcome on the podcast for having me You currently serve as Chief Investigative Officer at GoodReal. Uh, can we start by talking about how you got to this point and, uh, yeah, what, uh, what you've done prior to, uh, GoodReal? Yes. Uh, so prior to Get Real, I was a journalist for over 10 years. I worked in, uh, broadcast in, in the US for big broadcasters like NBC and ABC. really started my career in, uh, visual verification. So at the time it was called social news gathering, but it's essentially the ability to verify content that's circulating online. Ten plus years ago in journalism, that was just-- we started to understand that information being shared on social media was extremely valuable. Um, it was the first place where breaking news events would happen. It was the f- you know, everyone got one of these in their hands, and suddenly we had a lot more footage and photos to work with. Um, but that also meant that it needed to be identi- uh, authenticated. I built and managed teams that verified, uh, breaking news content, and I did, uh, investigative work as well. And I, and I ran my own YouTube channel at one point, uh, to help audiences how we verify content and helping them navigate, at the time, a complex online world. It is even more complex than it was five to six years ago when I had the channel, which landed me at ABC News, where I continued to help audiences figure out what's real, what's fake, um, giving them opportunities to understand the tools that were starting to develop. At the time, it was, uh, gen AI started to appear, and So I helped, uh, audiences understand what these tools were capable of doing, which eventually led me to Get Real, where I lead investigations and expert response, is, uh, essentially we at Get Real protect enterprises, governments, and individuals from AI-driven identity threats and deepfake attacks. I've always been in-- my mission has always been to help people and to help audiences understand what's real and fake, and that is continuing my mission, but from the technology side and working alongside, uh, scientists and researchers trying to build technology that will protect companies and, and help individuals spot, um, deepfakes. So would you say that what, what you do today at, , Get Real is, uh, is similar to, , investigative journalism? Um, I would say that it, that my experience understanding how information moves online, what kind of threat actors are out there, how misinformation spreads, um, all of that cumulative experience over the last decade, right? Figuring out, getting to a source, figuring out, um, an informa- where misinformation started or, um, who's behind, what is the intent behind a, a false piece of content. All of that has helped shape my work at Get Real. Um, but we are very much, I work hand in hand with, uh, forensic, forensic, uh, researchers and Hany Farid, who is the pioneer of the science. And, so we, we kinda have our, both of our expertise working together to help, um, companies figure out what they're looking at. And, uh, how do you compare, you know, when you started in this career trying to understand what is real, what is not in terms of online content, and when you compare it to what you've been seeing now at GetReel, how, much has the landscape, uh, changed over the years? It's changed tremendously. Um, when I first started, we were sort of naive about, you know, content circulating online. At the time, TV producers would just take an image or video and throw it up on the air, throw it up f- in their coverage. And soon enough, we realized can... are using old images, old videos during breaking news events to try and fool people and get, um, and get views. I mean, the the idea of, of, of, you know, scams, um, actors, of that intention has remained the same, but it's the tools and the technology at the disposal of those who are looking to harm or to fool audiences that has changed. So at the beginning, we were dealing with edited images or old images circulating, or, um, a lot of people pretending to be victims of a mass shooting, for example, or pretending to be a witness of, of a terrorist attack. Um, and fast-forward 10 years? we are, you know, now completely inundated with that is nearly impossible, if not completely impossible, to tell whether it was created by a human or whether it's synthetic. And it's not just either/or. It's really complex because people are using... becoming-- becoming second nature to use some of these, uh, generative AI tools. They're built into our phone, right? When we take a photo, we can edit it. We can remove an object. We can clean up a background. Um, all of these things that weren't so easy to do at the time, now that barrier to entry is completely reduced. And so there were techniques that we've always had in journalism to verify visual content and visual evidence. Um, I would say that today need technology on top of those, that skill set Maybe we could d-define, uh, our terminology at this point to make sure that everyone, everyone, uh, understands the-these words because you mentioned deepfakes, uh, you mentioned synthetic content. What, what are deepfakes exactly and how do we, how do you distinguish it from synthetic, uh, identities or synthetic content? Um, deepfake is really more of the popular term that most people are u- are, are used to hearing. Um, it, it dates back to 2017, when a user named Mr. Deepfakes-- actually, the, the root of it is, is quite horrendous, right? So the beginning of how this type of content started was back in 2018, where a Reddit user started to share, um, naked videos of actresses and swapping out of existing pornography and putting the faces of, of actresses. That's how the term deepfakes was coined. It later became popular vernacular to describe any type of content that was generated using a generative AI tool. So, um, some of, you know, many people will have seen Pope in a puffer jacket, uh, which was kind of the turning point when we think of what I call synthetic content or interchangeable with deepfake. It was the turning point for you know, for, I would say, a mass audience to start to realize these tools were capable of. And at the time, that was generated with Midjourney. I had found the person who created it, and it was already s- felt like it was so hyperrealistic. Jump to today, three, two to three years later, it, it's hard to call them deepfakes because the implications of them are so, can be so profound. Um, so we sort of interchange it with synthetic media, which is essentially created using any gen AI tool. Okay. And are, are there to- today, are there any reliable tells that, uh, uh, you know, content was, uh, is, uh, synthetically generated? I have a hard time answering that question because-- and I always have. And the reason why is that every time I did, a new model would come out and it would correct which, uh, whatever error, you know, the technology was getting at the time. At the time, it was very popular to say that there were six fingers, um, and that was a good tell that the, the image was synthetic, or look in the background and the faces were melted. Little by little with each mo- new model that came out, all of those things were corrected, not only in images, but video is getting extraordinarily good. Um, so every time I give a little tip, a little advice, a week or two later something comes out and it's corrected it. And I, I think what's me the most is that recently I would've said, well, if we're talking about replicating human beings, would've said, "Look at their skin. It's always overly polished. It looks a little waxy. They're a little too perfect." And now you have entire pipelines of, um, you have entire like workflows where you can add in imperfections to make sure that people can't, you know, your consumers or your users can't actually tell that created an entirely fabricated persona. Um, then there are ways forensically that we're able to, to... There, there's the technology and then forensically, if you look at the shadows often, um, and you know, the Hany will explain that is very good at mimicking physics but doesn't quite understand it, and so therefore all of the things that are-- it's replicating a physical world. But you can tell when you're applying certain forensic techniques that the image is synthetic. For example, it doesn't understand the vantage point. That's one of the things that we, that we look at on cases that are more complex. we can look at shadows and lighting. That's another one where it often fails. Is this something an everyday person is going to do? No, it's not. Um, but it is, it is a tool at our disposal for those who have a little bit more, who are more trained, who are trained as forensic scientists, et cetera. . I was just thinking of your video that, uh, GetReal published with these synthetically generated influencers, and it, it is really, yeah, like with a naked eye, impossible really to distinguish, have said the voice was flawed at some point, right? Like the voice always sounded a little robotic and now they've corrected it. And as I was making that piece, I was using a lot of content from one model called Kling, which at the time was like the most pr- w- was the most hyperrealistic. And then C Dance out with a new model, and that some of the footage that you see in there is coming straight from C Dance, and that another jump in realism. And when I mean realism, I mean the way that the, the synthetic, the synthetic person, the synthetic avatar moves in the space, right? The way that they talk, the way that they move, like the body language, of that is becoming really to reality. Um, there's some things that it can't do that's like very weird that they can't do, like singing. I don't know if you've ever seen an AI influencer, AI in- AI avatar sing. There's something really off with their tongue, like the, the movement of what their tongue is supposed to do, it, it, doesn't match at all the way that they're singing. So that's-- But you, again, you have to be like super attentive, right? Um, to those details. But that's kind of a tell in terms... So if the avatar is singing, they usually, they can't do it very well. So would it be a good, A good thing to do when , someone calls you impersonating another person that you know, some family member to asking them to sing? Well, it could-- Well, issue of that is that in real-time impersonations, it's often a, a real human body, you know, just wearing a different face mask. So it's not an entirely generated piece of content, it still retains some human properties. Like if I'm using clean motion control, not real time, but it's still based on a human motion. And then if you're doing real time, if you're changing your voice in real time or changing your face in real time, like I don't f- that you're able to s- you're able to still keep the characteristics of a, of a human Mm-hmm. I was also thinking as I was watching the video when I was, you know, trying to come up with, uh, some pos-possible strategies to verify if this is, uh, an actual person to use, uh, some facial recognition, uh, technology which could show me that, uh, - the entire digital foot-footprint of this persona is only one month old and I don't see anything historical. Is that a way to, to verify? that could definitely be a way. I mean, there's, um, although now they're very good. If you take some of the big influencers, they've been around already two to three years on the platform. Um, and then they're very good at repurposing old accounts that have existed for six years and filling it with content. Um, I'm always skeptical when they're trying to sell me something 'cause often than not, every single one of these accounts is trying to sell you, to drive you to buy something, right? Drive you towards buying vitamins, some he- some, like, bogus health hack, um, some e-book, some clothing store. Like, there's always a push towards consumption and, buying, and that's sort of one of the things that always raises a, a, a flag for me. And then the other thing I've noticed are the patterns that they, they're creating-- People are creating these accounts based on different niches. So they're really preying on people's, um, like, emotions and what they know is going to lead to views based on social media algorithms. So one of them is like, w- you know, a big niche on social media is self-care self-improvement. So they're creating these AI influencers and these AI avatars within those niches. So you're gonna have one big theme is, like, bounce back after a divorce. So I've seen, like, all of these fake videos, and then what do- does it push you towards? Buying some weight loss pill, supposed weight loss pill. And so in every single niche, there's, like, self-improvement. You have one that's, like, fake podcasts, um, giving you financial advice or marriage advice. Um, so every single category, health, um, fitness, you have now AI influencers created for the sole purpose of, like, farming engagement and selling you c- and selling you And , do you see only commercial applications for these AI influencers? Or do you also see them, you know, in the context of some state-backed disinformation campaigns? s- we see them in, uh, yeah, the political realm. So it-- but that's always been the ca- it-- that's what I meant at the beginning when I talked about there's a certain way where, you know, b-being, like, not... Like, bad actors have always done this, it's just that the technology at their disposal has changed. And so, or even the way that disinformation, propaganda, or narratives have worked online is the same, it's just the technology now in the hand. So f- one example is influencers. have used influencers on social media, paid them um, push a certain narrative or a certain point that they wanna get across in elections. Now, instead of paying actual influencers, they're just mass creating their own, creating accounts and, you know. There was this example with the MAGA influencer, uh, except that one was created by, I think, a s-student in India. Um, and that was also for the sole purpose of, of making money. Uh, but now there's evidence as well that some of the campaigns w- you know, and on both sides are creating their own accounts of, uh, influencers And do you think -- the ability to generate these synthetic, uh, synthetic, uh, social media accounts, really represents a fundamental shift in terms of, you know, let's say, in the past, this is just some, you know, uh, arbitrary numbers. I don't have any like, uh, statistics on this. But let's say in the past, uh, Ru-Russia could have influenced like 10% of the population of the, I don't know, Czech Republic to vo-vote, , in a specific way in parliamentary elections. Do you think that now, since the bad actors have access to this,, technology, this impact can be potentially much higher? Yeah, I mean, it, it's, it's will give any nation state and any bad actor the ability to mass create the type of content they were creating before. Uh, think about when the w- the war in Ukraine broke out. Um, one of the first videos that we call deepfakes was of, um, of, of Zelensky. It was created, Emily, by Russia announcing that they were going to retreat, telling the troops to go back home. They also, I think, hacked and got in on, on, uh, on, uh, TV and on the radio. y- Ukraine was very reactive and, and said immediately that it was not an, an authentic video, um, not to believe it, et cetera. But it did have this, like, initial impact, even though it was really terrible. Like, it was an, it was an awful deepfake. Think about what that would do today. Um, without the video, like, the voice is so good. Um, voice generative AI audio is so good at replicating any human being's voice that you only need three to five seconds of audio, and if you take even more, it's even more, even more believable. that were to leak, it creates enough panic and enough, uh, confusion, right, to an impact already. And now, yes, and, and the issue is that we're also... This is going to be distributed on platforms where we don't have visibility. If it's on Instagram, if it's on Facebook, like, eventually journalists will... get in the hands of a journalist, right? But when it's going through WhatsApp or through Telegram channels, like, that becomes harder to know how this content, where it's spreading, how it's being made, et cetera. Um, yeah, I can imagine that you can do a lot more harm now. And what is the solution then if you, you know, as you said, uh, earlier that you cannot really tell, , that a, that a social, , account has been synthetically generated with the naked eye? Is the solution just not to use social media and only rely on established, uh, media platforms? I think it, this is going to be true of every form of digital communication, that we are going to move towards a world where cannot have unverified conversations where we will not trust something that carries some form of verification. Um, you know, whether it's a mix of provenance information, so how has this photo or image been created or edited, whether that's using this C2PA standard or SynthID, which is Google's invisible watermark. We are going to start to only be able to rely on believing content that can show us how it's been created. I think that's the way the world is gonna move. And if we, uh, kind of focus more on the corporate, uh, sphere, how would you say businesses can, can safeguard against the, the, the increasing risk of, of deepfakes passing ID verification? , It's a, it's a, it's a complex issue, but are, for example, we're developing technology where, um, we can detect deepfakes or syn- the use of any synthetic voice or, or f- or filter, um, in real-time conversations. So on a Zoom, on a Webex, on a Teams, we can alert security teams or an HR team whether or not, uh, the person sitting on f- in front of you is, one, real, And whether or not they are who they say they are. Because it's not just a question of is this person real, it's also a question of are they who they say they are biometric information, and do they stay consistent within, for example, hiring interview process or throughout the company, or did someone come in and use their, their identity? Um, there, uh, I think the, the number one thing is already enterprises to be aware of the threat, to understand how these tools are capable of by- bypassing identity checks. Um, they're already doing it. Um, you know, there was re- recently a story, um, that, um, Instagram accounts were hacked using... I don't know if you saw this, but essentially, um, hackers claimed that they hacked, uh, thousands of Instagram accounts, including the White House, using the AI chatbot and just s- and just, um, telling the AI chatbot that they had lost their password and giving their information. And then if there was the second layer of verification, which is a, is a selfie, um, they were able to bypass it by taking the profile images and animating it using AI. And so a lot of these detection, uh, methods aren't, um, So the first step is making sure that systems you have in place are deepfake-resistant And are you aware, I don't know if this, maybe you have different providers in the United States. I think i-in Europe, uh, the most frequently used, platforms for remote identifications are IDnow and WebID. Let's say , if you wanna, uh, register on a, you know, car sharing app, they will only require, let's say they will use IDnow as a verification platform, and they will only require you to take a photo of your ID and then take a selfie and that's it. That, that sounds like that could easily be bypassed just given how, um, incredibly good, uh, the ability to generate a synthetic ID. Like there were videos circulating recently of Grok, from Grok, um, even from OpenAI, synthetic ID plus synthetic person. You can match, right? You can create like a believable synthetic ID plus a person. So if you're doing also even just a live a live verification, you can inject that video of a fake person holding up their fake ID and pass. Um, no, that, that would not be sufficient. So what, would be the, th-the solution , there has to be built-in deepfake detection, synthetic detection. Not just verifying biometrics, but deepfake detection, making sure the content isn't synthetic or the person isn't synthetic. And then going on to the second step of biometrics and v- and identity, identity verification. Okay, understood. Let me ask you this one last question. This is-- I know this is, this is also quite complex, but, . W-what would you say, uh, is the threat horizon for deepfakes? , How do you see it changing over the next three to, to five years? I think that we are going to see a lot more, um, attacks surface. Like, we hear about them because we are talking to CISOs and enterprises and governments, but I don't think we've scratched the surface of how this technology is going to be used to break into financial institutions, into governments, um, steal information, to steal funds, and to s- to scam the individuals. And at the end of the day, of the day, an individual is also an employee. And if they're vulnerable in their personal lives, they're vulnerable on your enterprise systems. so I think we've only seen, like, the tip of the iceberg in terms of, of the types of, of threat we are gonna start seeing. Thank you. uh, yeah, just something that people, everyone has to take, businesses, individuals have to take seriously Uh, the, it's, you know, I encourage anyone to take a look at these tools are actually capable of doing, um, with your own identity. I, yeah, I would be scared. I would be uncomfortable yeah. But a good place for people to start is to listen to this podcast and, uh, be informed. So Emmanuel, thank you so much for your time. I really appreciate, uh, this, and yeah, I wish you all the best Thank you so much