Why Social Engineering Still Works, Why AI is Making it Sharper, and the One Habit that Stops it
In early 2024, an employee at Arup, a global engineering firm, joined a video call with several colleagues, including someone who appeared to be the company's CFO. The call looked normal. The faces and voices were familiar.
He went on to transfer $25 million to several bank accounts in Hong Kong because the CFO on the call gave him the go ahead.
Every participant on that call except him was a deepfake.
That story made headlines because of the dollar figure and the production involved. But strip away the AI and what you have is one of the oldest attacks in the book: someone built a believable story, put a credible face on it, and waited for a person to do what people naturally do when authority figures make requests: listen and comply.
The Attack That Never Went Away
Social engineering has been the opening move of serious attacks long before anyone had the tools to clone a voice or generate a face. It works because humans are wired for cooperation. We're inclined to be helpful, to avoid confrontation, to give reasonable-seeming people the benefit of the doubt.
Security awareness training knows this. The techniques are well-documented: pretexting, phishing, vishing, baiting, tailgating. These aren't obscure. And yet the Verizon Data Breach Investigations Report consistently shows the human element showing up in the majority of breaches, year after year.
The problem is that knowing a tactic exists doesn't make you immune to it in real time. When someone walks up to your reception desk with a clipboard and a plausible reason to be there, the only decision being made in that moment is whether to be the person who holds up the IT guy.
Six Server Rooms and a Backpack
A few years back, I ran a physical penetration test for a healthcare organization. Multiple facilities, real staff, real security infrastructure. My job was to find out how far I could get without authorization. The goal: the server room.
The approach was about as low-tech as it gets. I posed as an IT repair person, gave a plausible reason to be there — printer issue, maintenance call, a ticket that had been submitted — and when someone asked who sent me, I gave them the IT manager's name. Which I found on LinkedIn in about 30 seconds.
Story, name, reasonable confidence, and a backpack that looked like it might have some tools in it.
At location one, I was waved through. No call, no badge check, just me standing in the server room with no one around. Location two, same result. Location three, same result. By the sixth location, I'd stopped being surprised by how easy it was.
The pattern was consistent: nobody wanted to be the person who held up the repair guy. Nobody had a clear, practiced process for what to do when someone showed up and their story felt reasonable enough.
Location seven stopped me. Not because they detected anything suspicious, not because my story had holes in it. They just said: "Let me verify with someone before I take you back." One call. That was the only thing that changed.
Six out of seven server rooms. The one that held wasn't running a more sophisticated security program. They just had a habit.
What AI Changes and What It Doesn't
The penetration test relied solely on confidence and a borrowed name. It worked because humans are naturally bad at real-time verification and good at not wanting to create friction. That's always been true, and it's still the core of every social engineering attack.
What AI changes is the production quality of the pretext.
Voice cloning tools can now replicate a specific person's voice from 10 seconds of publicly available audio. A podcast appearance. A conference talk. A clip from your company's social media. The output isn't a rough approximation. It's convincing enough that the person receiving the call has no technical signal to doubt.
That's what made Arup possible. The concerning version for most organizations isn't the elaborate multi-person deepfake conference call. It's the one-to-one voicemail. An employee gets a message that sounds exactly like their manager asking them to process an urgent payment, reset a vendor's credentials, or approve access for a new contractor. The attacker doesn't have to be in the room. They don't have to be good at improv. The model handles the voice.
The playbook from that healthcare engagement (show up with a name and a story) now extends to phone and video in ways that are genuinely harder to detect. The manipulation is identical. The tools are sharper.
The Defense Is Simpler Than You'd Think
Location seven didn't stop me because someone recognized an attack pattern or ran through a mental checklist. They stopped me because they had a default behavior: verify before access.
The organizations that hold up under social engineering pressure aren't necessarily the ones with the most sophisticated training programs. They're the ones where verification is normalized, where the process is clear enough that people follow it without feeling like they're doing something unusual.
A few things worth building:
Verify through a known channel, not the channel the request came in on. If a call sounds like your CFO asking for an urgent wire, you don't verify by asking a follow-up question on that same call. You hang up and call the number you already have on file. The same logic applies to email, voicemail, and video. The channel a request arrives on shouldn't be inherently trusted.
Require out-of-band confirmation on high-stakes actions. Payments, credential changes, access grants. These should require a second, independent confirmation. This is the friction that AI-assisted impersonation struggles with most, because it forces the attacker to compromise a second channel they may not control.
Treat urgency as a reason to slow down. Social engineering depends heavily on urgency. The CFO needs this processed before the call ends. Unusual urgency should trigger more caution, not less. A culture where that instinct is normalized is the right target.
Update what suspicious looks like. Awareness training that covers phishing emails but not voice cloning or physical pretexting isn't covering the threat as it exists today. The examples people internalize need to reflect the tools currently in use.
What Location Seven Already Knew
The Arup story and the penetration test have the same plot. Someone with a credible-seeming identity made a request, and a person in a position to say no said yes instead.
One used deepfakes. The other used a backpack, a LinkedIn search, and a reasonable amount of confidence. Both worked for the same reason, and both were stopped by the same thing: a person who had a habit of verifying before acting, and followed it without thinking twice.
The attack surface is human. The defense has to be too.
