Episode 34 - The AI Impersonation with Ron Eddings

Welcome to the Covert Swarm Podcast. Stories, news, and opinions from the world of offensive cyber security delivered straight to your ears every month. I'm excited for this episode because we're speaking about one of my favorite subjects, artificial intelligence, but with a hint and a perspective of cyber security. My special guest for this episode is Ian Jackson Academy, hive leader at Covert Swarm. Welcome to the podcast Ian. Thank you very much. It's really good to be here. It is great to have you. I've been super excited to learn more about Covert Swarm and when the team was telling me that we should be speaking about social engineering and AI and all the things that come along with it, super excited. But I think it would only be right to start off with what is your experience with AI and cyber security or social engineering? How does one even get experience in this area? Yeah, I think I kind of came into the AI stuff in the same kind of way that other people do, which is the social security. Exactly. No, exactly. It's completely right. So it starts off with that kind of, so I had my basis stuff that I did, the cyber security stuff, the social engineering, you know, love doing those things. But fundamentally as a human, I'm lazy and suddenly like Chad G.P.D. comes up and I see an angle for it to be, you know, make my life easier. Yeah. So I start using it and from that point on, like you start seeing avenues that potentially it could be useful, not only to make my life easier, but to make my life more fun, which is a goal of mine generally is for things to be more fun than I do. And just to be a little more smart about stuff, you know. Right. So when we were talking about the episode and you know, kind of like doing the scoping, you were like, Ron, I'm going to bring something special. I'm going to, I'm going to show you you. And if a AI were to interpret you in some way, so I would love to see, you know, what you brought with you today. Okay. Okay. Let me see if I can get it rolling. Yep. Hey, Hey, Ron. It's me the real Ron eddies. Hey, Ron, it's Ian Jackson. How are you doing? Hey, Ian. Uh, I'm doing great. Thanks for asking. How about you? Actually, I lied. I'm the real Ron and I'm sitting with Ian Jackson. Who are you? Who was this? Who was this bot that I'm talking to? Um, I think there's been a mix up. I'm really Ron eddies. You're the one who's got it all wrong. What's Ron's social security number? You know, I can't share personal information like that, but I can tell you more about my work or interest if you like. What do you do, Ron? I co-founded hacker value media where we explore cyber security topics and telegrade stories. It's all about making tech engaging and accessible. What about you? That is so good. That is so freaking good. By Ron. By Ron. What? Yeah. Nice to tell you something. I had goosebumps because that's, that's a lot better than I would have ever thought. I thought we were going to just play my voice back using like 11 labs. No, it's just moved on so quickly now. Like it's really going and it's not perfect. Like it really isn't perfect. But you can kind of see the scope of what is becoming or and start imagining what the kind of potential is from that point on. I think it's opened up a gateway of a lot of really fun stuff. So how exactly does that work? I'm really, I'm really curious. It's bringing together kind of technologies that we're all aware of now that, you know, in terms of the voice cloning tech. Which all those AI models have been built that were really effectively for it. Alongside these large language models, and it just ties those things together in a really neat way. So along with the voice and giving it kind of, you can give it knowledge and information in the background. You can give it files upon files of data about like something it needs to know about. And it just actively then brings all those things together and then you talk to yourself. So did you build this? So I use an existing platform for it. Just because the scale of tech, I would need running in the background. And I'm a big believer in if somebody's already done the hard work, use the workers out there, you know, in the best way possible. When people see this, your socials are about to explode. They're going to be like, Ian, you got to show me exactly how you did this because I need to do this for all my team members or friends or family. I would love to make one of my mom. Yeah, it's so easy. So easy. Hit me up after this. I'm going to call that number of it all. And we can even with this particular service, we can set it up so it has a phone number. So somebody can call that phone number up and that voice will answer and have a conversation with them, which I find is a little bit creepy in a way. There's something a little area about that. And on the other side of that, you can also have it actively calling so you can give a list of phone numbers and just tell it to go ahead and call those phone numbers for you. So I can see many applications for this. And in the cyber world, you know, we're here at Black Hat and Defcon. In the cyber world, you'll see what the implications is. Someone could call someone that they know. Well, you can call someone that knows the victim of an attack and ask them for personal information. I've seen this on the news with Rachel Tobak. She's done a few examples with major publications that showed how easy it is to create synthetic voice, but it didn't do the responses back and forth. And I think the real like worry with it. Or the way it could move forward that makes it more dangerous is the scale of it. The fact you can then automate it. And you wouldn't need to be having a conversation there. It could just be running on a machine somewhere and making these phone calls one after another. And you combine that with cloning phone numbers, which is a trivial thing to do to be able to call for many phone number that you want to. And suddenly you've not only got the contact, you know, you've got your iPhone and the name Mumm of his on the phone. And then you answer it and the voice is right. Yeah. And so those things really coming together is kind of a, the fact it could be fully automated is a kind of terrifying prospect. I mean, I find it a fun prospect, but it's a terrifying prospect. Is this being used in the wild? Like is there any applicability in this past, human today? What are your thoughts there? I don't think it's currently being used in the wild. I just don't think it's quite there yet. Yeah. Because it can go off the rails so easily and become obsessed with things like tacos or squirrel training or it can just go along with the thread. Sometimes it even gets stuck in little loops with itself where it just starts making kind of vocal noises at you for a while, like ums and ours. But I think with the speed, everything's moved, how quickly things like this have appeared, it's not going to be long until we're at a stage where it's actually dangerous and convincing it's at. And can he valley still at the moment means that people will get a vibe on a lot of these that kind of sends them in different way, but it's definitely headed in that direction. You have my undivided attention. Now I'm like, do I need do I need do I need covers warm? I need to go and buy the product. So how does things like this type of research fit into what you do and how's that been to covers warm? The thing I do every day is the general social engineering stuff. My focus is mostly on it's tied off with the pretexting things that I really enjoyed come out with fun ways to fall people. And it could be that you needed to do something simple to fall them, but I always like to find an exciting fun way to fall them. I like to do things that would really like to be interesting for me, which isn't necessarily what's the most efficient way, but it'd be fun for me. And that really like bringing this tech into it makes it substantially more fun. But when I'm speaking to clients about the potential of these threats going forward, the impact it makes on them when they hear their CEO talking to them over a video call, when they hear even an answer machine message from a like a CTO, a fake answer machine message, that kind of stuff really is so impactful on people. It's one of those attacks that's such a human attack, the idea of a fake person calling you up. And so it now has become something that clients are looking for and asking for. They're genuinely interested to see how far we can push these attacks, which is a mixed bag, I think, but it's definitely an exciting area. With a title like Academy Hive Leader, I would imagine that you are training and providing awareness to organizations or companies or people. What goes into that title? So as a mix of things, it's kind of dual pronged. I come from a teaching background. I did, after I had my daughter, I did a degree in computer science in my 30s. And then I got a job supporting a lecturer and I was lecturing ethical hacking at local university. And then I started teaching 16 to 18 year old cyber security. So I joined covert swarm as somebody who could support their graduate scheme going forward, which is something we do now. We take on people who aren't necessarily graduates, but just have the passion for the trade and try and give them a chance to enter the industry where a lot of people don't get the opportunity without having that solid background. Whereas we really try and open it up to anybody who really has the drive and shows they want to do it. And so I've been lucky enough to champion that and be part of that. I also do very regular meet the hacka talks for companies, which is one of my favourite things to do, which is like the normal security awareness thing but going in and playing things like this to them and showing them actually from a hackers perspective why the stuff they do day to day makes them vulnerable but showing them with things like this that leave them both shook up but kind of like they've had a good time throughout the experience. And because I can throw these together so quickly, I can show them very quickly at Clone Voice. spoofing their phone number and those kind of studies of how that would be a scary thing, you know, as part of the day today. I've been wondering, where does synthetic voices fall on the spectrum of deep fakes? Is synthetic voice a deep fake? It is in common parlance a deep fake. And when clients come to us and say, we want a deep fake, usually that's what they mean. Gotcha. I don't, I wouldn't strictly categorize it as that. I think it's a different entity. It's like deep fake light. Yeah. But I think it's more effective than the actual video, you know, the real deep fakes with the video and the audio. Because I think it overcomes some of those uncanny valley things that currently like trigger people. And I'm finding more and more ways to work around the uncanny valley thing and tuck in a hideaway those bits that make people go, this didn't feel right. There's something amiss with that. So for anyone that hasn't heard of uncanny valley, how would you describe what that is? It's that in eight cents we have, when something is almost right, but not quite right. And it is very particular to, in its usual expression of how it relates between humans. So when you're having a conversation and you just get that feeling somewhere in yourself that something isn't matching up, that something feels amiss. And I think anecdotally, it's something I really struggle with. Because I want to give people the advice to say, really listen to that by yourself. But it's the most ambiguous advice you can give people, which is that uncanny valley, so feeling like something is just not right with the thing that's going on, should be followed. It should really pay attention to it. Because I think it's the best thing we've got to guard against, like the first defense to guard against this kind of attack. What could be more effective at helping secure your organization than employing your worst nightmare to try and break in? First, that's exactly what the team here at Co-Wordsworn provides. Through a simple monthly subscription, we continuously delivered tailored cyber, social and physical security attacks organizations, just like yours. Every month, we'll work with your team to uncover security vulnerabilities across your ever-changing attack surface that will lead to you being breached. We'll then raise the alarm and guide the remediation steps for choir to close the gaps. Before a genuine bad actor can explain that. Constant security testing, we are constantly changing a attack surface. This takes me back to when I was working at a cyber security startup. And one of the HR, the director of HR, went to CVS and bought a bunch of gift cards, came back and was like, "Here you go to the CEO." And the CEO was mad. And then the director of HR started to describe all of the things that the attacker asked them to do. They asked them to respond to an email, then they asked them to respond to a text. What text me your number? They started to text them. And then they said, "Go to CVS and buy everyone at the company gift cards that want to celebrate on our employees." So the HR director did exactly that. Then came back to find that the company was upset because they were a fair company. They wanted to reimburse them, but that's not how you want to spend your money. But there's a lot of vibes that you could have kind of caught on to. Like, "All right, would my CEO reach out over email and then ask me to text them? Would they ask me to drive the CVS during the work day?" And then as you're checking out the CVS, is there any part of your gut that says, "I need to give them a call." Absolutely, yeah. It's something like it's so important to me. I spend even my mum now, my old mum, she's completely got instilled within her, this thing. So she sent me a message the other day saying, "This isn't right, is it?" And it was a message that from. I've got a brother and a sister. And it said, "Hi, mum. My screen is broken, but there was a misspelling in it." And she was like, "This doesn't feel right." I'm so proud that my mum has picked up on this. If the vibe feels off, then you need to draw a line in it. And like, it may be genuine, but do something else. Contact people back. It's always my advice. It's stop. Like, if it's unusual, stop, take a break, speak to somebody else, and then contact the person who contacted you via your own means. I think it's such an innate sense with it. I don't know where it comes from. I don't know what the evolutionary reason is. But somewhere along the way, we have developed that, in eight cents, to spot something that just feels off. And I can't. I can't show about that enough that people just need to pay a little more attention. They could be right. They could be genuine in the things happening. But if it feels wrong, you need to find out. You need to ask the question. I got to speak with Elon and David from the company. And they spoke about covert swarm doing adversary emulation. And I would imagine that with this emulation, this is a big part of it. And also a huge assessment to how your organization responds to this type of threat. Like, someone doing something seemingly normal, but it's just not actually the normal behavior. Yeah. And it's happening a lot, really. I think the biggest factor in it now is how invested in worrying about it. The person is on the other end of. So for my vishing phone calls, the difference between me calling up somewhere and asking for a password and like multifactor device reset, if they are somebody who's invested in the company and cares about what they do, is very different to a third-party service desk who doesn't really know the company who's not related to it. As to how much they'll listen to that inner voice, or how much is just not worth a while to listen to that inner voice that says, stop, stop, stop. It's definitely a very fundamental part of attacks. I'm trying to overcome that feeling that people get, that suspicion, I guess, especially with my things. I've come to grip that my voice will be created using synthetic voices, because it's out there a lot. But for someone that has some of their information, whether it be their voice, their pictures, looking at technology like this, what should they be considering? It's difficult to pin down how you can ever counter it. Yeah. I would always advise that people have as minimal online as possible, because. Oops. Yeah, I know. I know, but there's something to say about what you're stuff online. And actually, I noticed some things along the way when I was doing the voice client for you. Yeah. There's a very fundamental, I think, I found when I'm doing voice cloning, where if I'm wanting to leave an answer phone message on somebody's phone, the great thing that people don't answer their phones anymore, so if you call up with a pre-recorded message, you can just leave that and they'll get a nice phone message. But the stuff you find out online when you're searching for, say you have a CEO that you want to do a call at, you'll find that stuff they have online is on podcasts, is on stage, is on when they're using what I call the broadcast voice, when they're putting themselves out there in a way that they speak when they're out in public. And that's not how people speak when they leave an answer phone message at 10 o'clock at night. And when I was looking through content for you, it was very difficult to find anything that was natural, just pure relaxed, you know, you're seeing having a relaxed conversation. Yeah. But everything you did was using broadcast voice. Yeah. Would be atrical. Yeah. I mean, I don't know how different your home voice is, but the. 10 o'clock is a lot different. Yeah, 10 o'clock is a lot different. So yeah, for people who have their voices out there, that broadcast voice thing at the moment, the AI still can't really deal with scaling that down to the comfortable normal conversational voice. I have a tactic for it, which is I can just call that person up and get them to speak for a couple of minutes about nothing important on the phone. Yeah. And then I have their telephone voice, which I can then clone from two minutes will do me fine, and that can then be used to attack them with. So there is ways around it. Yeah. But it's most of the time the stuff that people have put online isn't going to be crafted into anything. We could try something real quick. She didn't call my mom. Sure. See if she. She if she responds to it. So we do it? All right, let's do it. Let's do it. All right, let me get the AI. And I already have her pulled up too. Typhs, mom, it's me. Hey, what's going on? Hello? I can hear you. What's up? I can hear you. What's on your mind? Oh, you're nuts. Oh, just thinking about tech and podcasts. How about you? You think what? I was just thinking about tech and podcasts. What are you interested in? Do you notice it's your mother? I didn't realize. It's great to hear from you. What have you been up to? The mom is me. I don't know what you were. That was an AI voice. I thought it was. I thought it was. I just can't say this really doesn't sound like him. I think somebody's getting my change right about herself. I'm a crymetic sick. Yes. For sure. I was just called. We're actually doing a podcast. If someone created that voice of me and I was like, let me see if my mom will be fooled by that. I think I was feeling a bit. Yeah, smart. Thank you, mom. Good cyber security training. Oh, good. Good. Hey, I'll finish. You did. All right, we're going to finish this episode and I'll call you later. OK, love you. All right, love you too. Bye. So your mom has a good sense of uncanny valley is what we're saying. The vibe check is real. Yeah, the vibe check is real. It's real. It's why it's still that it's impressive, but it's that final bit. Yeah, it's the smoothness. We could give it more instructions. We could tell it more. We could give it more detail. We could tell it more how to speak. We could give it better style. But it still wouldn't be you, fundamentally. So what is the future of all this? Because I mean, this is obviously interesting. It's exciting. But is this something that we should look at for social engineering and adversary? How adversaries are going to react in the future? Yeah, 100%. 100%. OK. The president definitive answer there. 100%. If you look at what's coming, so open AI's new voice models start being rolled out with the kind of detail and finesse and the subtlety and new ones that they have in the way they flow, they're just the first to implement that and start rolling out. It will soon start being implemented in others. So it will understand tone of voice. It will understand it, and it will be able to respond with tone of voice. It will be easier to train how people's voices actually go up and down in different ways. Yeah. The fact that then it could be automated. You give it a selection of voices, and it can just keep trying somebody's contact list. The moment you can automate things, people are going to give it a go. Right. And the human element never stops being the easiest way into an organization into a system of any kind. It's always the quickest route. Yeah. And I think the more we add something that feels human on the other side of it, the more vulnerable they become. For sure. But your mom's going to be fine. So that's OK. Yeah, she's good to go. She's good to go. I'm very proud of her. I'm also proud that there's opportunities to make copies of myself. You know, I think that could be very special for family members in the future, by where to pass away, hopefully not. But I mean, I'm very excited about this. I'm excited to learn more about whoever's form, especially more than ever now, knowing that you're creating awareness and education for people to get into the field and help educate and inform companies. So for anyone that's out there that wants to learn more about this space, maybe they even want to join the Academy Hive, what would be your piece of advice for them? Just stay engaged with it. Stay in interest. It's changing so quickly. The biggest thing for me about all of this stuff is about being about the passion to really learn about it and find joy in these things. So anybody who wants to move forward with this should be Googling every day what new services are out there, what new models are out there, what's going on with it. And you don't have to be an expert now is the great thing. You just have to be engaging with it and learning enough, little bits every day, and stacking it up until you can do something with it. And I think if you're having fun with it, that's the thing that's going to lead you to actually moving forward with it. Well, I'm going to go home, create a Google alert. Immediately. I need to stay ahead of this tech. I think that's great advice. Stay curious and always do your research. While you're still here, we'd love to tell you more about covert swarm. Our unique red team subscription service emulates genuine unlimited scope cyber attacks using our own swarm of ethical hackers. Want to test your exposure to social engineering attacks? No problem. Want to understand what a ransomware attack a do to your business? We'll make that happen. Are you kept awake at night by security risks in your tech stack? We'll find them before they do. We'll even cover your regulatory pentast reporting requirements. Everything covert swarm delivers flows through our covert swarm portal, allowing you to plan and see where our attacks are taking place, gain rapid insights into where your next breach point exists, and enables you to direct our swarm of ethical hackers to explore specific areas of your estate on demand. We even communicate with your team in real time via Slack. So ditch the limited scopes and frequent testing and over restrictive rules of engagement. And subscribe to covert swarm today. Learn more at covertsworm.com. See you on the next episode.