I was sitting in a restaurant overlooking Annapolis with a few friends who’d arrived for the season. We cruising couples had collected in one place to refresh our friendships and feed our bellies. One of our friends ordered a nice red wine, and the server, a twenty-something with enough tattoos to make comic books obsolete and enough piercings to scare the MRI techs at the nearest three hospitals, asked for her identification. The friend pulled another friend’s ID out of the window of her folding wallet and handed it over. The waitress looked at the birth date and handed it back. One of these friends was substantially more melanin-challenged than the other. The waitress didn’t care whose identification she got. She cared that she got some identification. Now imagine me scribbling down the details to use some day in the future. You just can’t make this stuff up.
That scene captures something fundamental about how identity verification actually works in the real world. The check was never about confirming identity. It was about satisfying a process. The server needed a document with a date on it. She got one, so the process succeeded while the verification failed, and nobody noticed, because the process was never designed to catch that kind of failure.
Now scale that to enterprise authentication. Your systems are doing the same thing the waitress did. They receive a credential, check its format, confirm it matches a stored value, and grant access. The credential itself is the proof, whoever presents it.
That worked when the cost of producing a convincing credential was high. Forging a driver’s license requires equipment, materials, and risk. Guessing a password from scratch takes time and compute. Impersonating a voice over the phone requires a gifted liar who can maintain character under pressure. Each of those constraints imposed a natural limit on the volume of identity attacks any single adversary could run. AI removed those constraints in live operations. The credential stuffing attack that used to require a human operator reviewing breach dumps and building custom wordlists can now be optimized by a model that learns password mutation patterns from billions of leaked credentials. The voice impersonation that used to require a natural talent can now be synthesized from three seconds of recorded audio. The fake identity document that used to require a skilled forger can now be generated by a tool that produces documents indistinguishable from genuine ones at first inspection. The cost of producing a convincing identity claim collapsed, and the systems designed to verify those claims have not adapted.
Threat Model
The adversary in this chapter is not targeting your encryption or your firewall. The adversary is targeting the moment your systems decide someone is who they claim to be. That moment is the point of maximum leverage, because everything downstream trusts the answer.
This attack surface has four distinct entry points, each exploiting a different assumption about what constitutes proof of identity.
AI-optimized credential stuffing is the most mature of the four. Traditional credential stuffing takes a list of stolen username and password pairs from one breach and tries them against other services. It works because people reuse passwords, and it has worked well enough that Verizon’s 2025 Data Breach Investigations Report identified stolen credentials as the leading initial access vector for the second consecutive year, accounting for 22% of confirmed breaches. The 2026 edition moved vulnerability exploitation into first place, though Verizon attributes part of that shift to splitting pretexting into its own category. What AI changes is the sophistication of the guessing. Tools like PassGAN use generative adversarial networks trained on leaked password sets such as RockYou to learn how humans actually construct and mutate their credentials, and targeted-guessing research goes further by using a person’s own previously leaked passwords. The model doesn’t just try “Password123.” It learns that a user whose 2019 password was “Summer2019!” probably changed it to “Winter2023!” or “Summer2024#” and generates those candidates first. It correlates breach data across multiple leaks to build per-user profiles: this person tends to capitalize the first letter, append a year, and use a single special character at the end. The attack becomes less about volume and more about precision. IBM’s 2025 Cost of a Data Breach Report priced compromised-credential breaches at $4.67 million with a 246-day average containment window, which tells you how long these attacks go undetected even after they succeed.
The infrastructure requirements for this attack are modest. The adversary needs access to breach dumps, which are commercially available and growing. SpyCloud’s 2026 Identity Exposure Report documented 65.7 billion identity records in its recaptured data lake, including 642.4 million credentials from 13.2 million infostealer infections in 2025 alone. HaveIBeenPwned absorbed the Synthient Credential Stuffing corpus in November 2025, adding 1.3 billion unique passwords, of which 625 million had never previously appeared in the service. The adversary needs compute to run the GAN and a botnet or proxy network to distribute the login attempts. All of this is purchasable. None of it requires specialized expertise.
What does this attack produce that a defender could observe? Login failures. Lots of them, but distributed across source IPs and spread over time if the attacker is disciplined. Successful logins from unusual locations or devices. Session establishment patterns that don’t match the legitimate user’s historical behavior. These signals exist in your logs right now, but they’re buried in the noise of legitimate failed logins, password resets, and VPN connections from hotel Wi-Fi.
AI voice cloning against verification processes is the attack that matured fastest. CrowdStrike documented a 442% surge in voice phishing between the first and second halves of 2024. Deepfake-enabled vishing specifically jumped over 1,600% in Q1 2025 compared to Q4 2024, according to security firm Right-Hand. The attack template is straightforward: the adversary clones the voice of an employee from publicly available audio (an earnings call, a conference talk, a podcast appearance, even a LinkedIn video), then calls the help desk and requests a credential reset or MFA reenrollment.
This is the playbook that Scattered Spider used against MGM Resorts in September 2023, costing MGM approximately $100 million in a single quarter. The same group hit Caesars Entertainment the same week through their IT support vendor, extracting a reported $15 million ransom payment. Clorox was breached through the same method via their outsourced help desk provider, and later alleged $380 million in damages in a July 2025 lawsuit against that provider, Cognizant. In February 2026, the SLSH group was caught actively recruiting people with specific voice profiles to improve their success rate against help desk staff, offering up to $1,000 per successful call. Palo Alto’s Unit 42 documented a social engineering intrusion that went from initial access to domain administrator rights in under 40 minutes, and Mandiant’s M-Trends 2026 ranked voice phishing as the second most common initial infection vector in 2025. The UK Cyber Monitoring Centre estimated the 2025 M&S and Co-op attacks at between 270 million and 440 million pounds sterling in losses.
The operational constraint that mattered here was the rarity of talented human impersonators. That constraint is gone. Modern voice cloning tools produce a convincing copy from as little as three seconds of recorded speech. The attacker doesn’t need to be a gifted liar anymore. The attacker needs a laptop and a sample.
Deepfake video against identity proofing targets the face-matching step in remote onboarding and high-assurance verification. The Entrust 2026 Identity Fraud Report, analyzing over one billion identity verification events across 195 countries, found that deepfakes accounted for one in five biometric fraud attempts. iProov’s 2025 threat intelligence report found that face-swap attacks grew 300% year over year, and native virtual camera attacks (where the attacker pipes a synthetic video feed directly into the identity verification pipeline, bypassing the physical camera entirely) grew 2,665%. The attack bypasses liveness detection because the synthetic face moves, blinks, and responds to prompts. It bypasses human review because deepfake video quality now exceeds what a human reviewer can reliably distinguish. The Surfshark study drawing on public incident records puts documented global deepfake fraud losses at $3.7 billion, with 89% of that recorded in 2025 and the first half of 2026.
For a small or mid-sized company, this threat shows up in two places. First, any remote identity verification process that relies on a video selfie or a live video call is vulnerable. Second, the same technique can be used against business processes that use video calls for high-value authorizations. The Arup Hong Kong case in January 2024, where deepfake video of executives on a conference call authorized a fraudulent transfer, demonstrated that even multi-person video verification can be defeated.
Synthetic identity creation is the slowest of the four attacks but potentially the most damaging because it establishes persistent access under an identity that passes every verification check. The adversary uses AI to generate complete identity packages: a face that belongs to no real person, documents that pass automated verification, employment histories and social media presences that establish a plausible backstory. TransUnion’s H1 2025 fraud report found $3.3 billion in U.S. lender exposure, as of the end of 2024, to suspected synthetic identities across auto loans, credit cards, and personal loans. The Shufti Identity Fraud Index projects a 495% increase in deepfake identity fraud in 2026 over 2025, with document deepfakes expected to grow 40 times over 2025 levels.
In an enterprise context, synthetic identity creation threatens any process where external parties establish accounts or access: vendor onboarding, contractor provisioning, customer registration, partner portal enrollment. A synthetic identity that passes initial verification and then sits dormant for months before activation is nearly invisible to traditional controls because there is no real person to compare it against. The identity looks legitimate because it was built to look legitimate from the ground up.
All four attack types share one structural feature: they target the verification moment, not the authentication mechanism. MFA, password hashing, and identity proofing all keep working. The attacks succeed because the evidence presented at the verification moment, the credential, the voice, the face, the document, is no longer reliable proof of anything.
Traditional Tools
Take an honest look at what most small and mid-sized companies have deployed against identity threats, and you’ll find a program that was competent five years ago.
Multi-factor authentication sits at the center. For companies in the 500 to 1500 employee range, this usually means Microsoft Authenticator or a comparable push-notification app, sometimes paired with SMS fallback for users who resist app enrollment. The better-funded programs have hardware tokens for privileged accounts. MFA was the single most impactful identity control of the past decade, and it genuinely stopped a large class of attacks. Microsoft’s Digital Defense Report documented 99% effectiveness against automated password attacks at the per-attempt level. That number is real but increasingly misleading, because it measures protection against the attack class that AI is replacing, not the attack class AI is enabling.
Identity governance platforms handle joiner-mover-leaver processes. When someone joins the company, they get accounts. When they move roles, their access should adjust. When they leave, their accounts should be disabled. In practice, at companies this size, much of this runs through manual IT processes backed by an HR trigger. The governance platform might be SailPoint, Saviynt, or a homegrown PowerShell script that reads from an HR extract. Provisioning is usually faster than deprovisioning, because new hires call when they can’t log in but nobody calls when an ex-employee’s account is still active.
Account lockout policies cap failed login attempts before disabling the account, typically five to ten failures within a short window. These were designed for brute force attacks where the adversary hammers a single account with rapid-fire guesses. Rate limiting on login endpoints serves a similar function at the network layer.
Help desk verification procedures vary widely, but the common pattern involves knowledge-based questions: employee ID, manager’s name, last four of Social Security number, department, or recent login location. The help desk agent reads from a script, the caller answers, and if the answers match, the agent processes the request. The verification script was designed for a world where the caller is either the real employee or a stranger guessing, and the questions are designed to distinguish between the two.
Password policies enforce complexity and length requirements at creation time. The better programs screen proposed passwords against known breach lists using the HaveIBeenPwned API or a commercial equivalent. NIST SP 800-63B-4, finalized in 2025, now says verifiers shall not require periodic password changes, but many organizations still rotate privileged account passwords on a schedule. Password managers are deployed at some companies, but in my experience adoption stays low without aggressive enforcement.
Identity proofing during onboarding typically involves document verification for regulated industries (CMMC, HIPAA, financial services) and little more than an email confirmation for standard enterprise accounts. The onboarding process confirms that the person presenting documents matches the documents. It does not continuously verify that the account continues to be used by that person.
SIEM and log aggregation platforms collect authentication events, but identity-specific analytics are usually limited to canned reports and threshold-based alerts. Failed login alerts fire when a single account exceeds the lockout threshold. Impossible travel alerts fire when two logins for the same account appear from geographically distant locations within a short time window. These rules exist and work against unsophisticated attacks, but they generate enough false positives from legitimate VPN use, mobile hotspots, and cloud proxy services that many security teams have tuned them to the point of near-silence.
None of these tools are bad. Most of them are doing exactly what they were designed to do. The problem is what they were designed for.
The Gap
Every tool in the previous section relies on the same foundational assumption: that the credential presented at the verification moment is hard to forge. The password is a secret only the legitimate user knows. The voice on the phone belongs to a human who is either the real employee or a stranger who sounds nothing like them. The face on the video call is a live human face. The identity document was produced by a government agency. The answers to the help desk verification questions require knowledge that only the real employee would have.
AI removed the difficulty of forgery across every one of those modalities at the same time.
Start with passwords. Account lockout policies and rate limiting were designed for an adversary who guesses randomly or works through a dictionary. Against that adversary, five failed attempts followed by a lockout is an effective control. Against an adversary running a model trained on large leaked password corpora that generates high-probability candidates tailored to a specific user’s mutation patterns, the first guess might succeed. The lockout never fires because the attack doesn’t trigger it. The attacker is predicting the password from data, and prediction at scale against a breach corpus of 65.7 billion records is a fundamentally different problem than dictionary brute force against a single account.
Move to the help desk. The verification script asks questions the attacker can answer from open-source intelligence. Employee ID, department, manager’s name, start date, badge number, and recent login location are all discoverable through LinkedIn, corporate directories, social media, and previous breach data. The script was designed for a world where gathering that information took meaningful effort and where the caller’s voice provided an implicit additional factor. Voice cloning removed both constraints. The attacker arrives at the call with all the right answers spoken in the right voice. The help desk agent has no mechanism to detect that the call is synthetic. In Pindrop’s analysis of more than 1.2 billion calls, deepfake activity rose 680% year over year during 2024, with a 475% rise in synthetic voice fraud in the insurance sector alone. The help desk agent is structurally designed to be helpful, under pressure to resolve tickets quickly, answering calls from employees they have never met, and holding the keys to reset credentials and disable MFA. Those traits are the job description, not a flaw in the agent.
Look at identity proofing. Document verification and face matching were designed for a world where producing a convincing fake document required physical forgery skills and where a face on a video call was reliably attached to a live human. Both assumptions failed in 2024 and 2025. AI-generated identity documents now pass automated verification checks. Face-swap and synthetic face technology has progressed to the point where deepfakes account for one in five biometric fraud attempts. Liveness detection, which asks the person to blink or turn their head to prove they are real, is defeated by synthetic video that blinks and turns on command. The proofing moment, the point in time where the system decides this person is real, has become the weakest link in the identity chain because it is a single-point-in-time check against evidence that can be fabricated at low cost.
And look at authentication events in your SIEM. Impossible travel detection fires when the same user logs in from New York and Singapore within ten minutes. It does not fire when a stolen session cookie is replayed from a geographically plausible location, because the authentication event looks normal. The alert logic is based on physical impossibility, not behavioral abnormality. When phishing kits such as Tycoon 2FA use adversary-in-the-middle techniques that steal authenticated session tokens without triggering a new MFA challenge, the attacker inherits the legitimate user’s session and generates no anomalous authentication events at all. The SIEM sees a normal login because, from the system’s perspective, it is one.
The unifying gap across all of these controls is temporal. Every traditional identity tool verifies once and trusts continuously. You authenticate at login and your session is trusted for hours. You verify your identity at onboarding and your account is trusted for years. You call the help desk, answer the questions, and the agent processes the reset. Each of these is a point-in-time verification event, and the trust it grants persists long after the verification moment has passed. AI attacks exploit the gap between those verification moments. The credential is stolen after authentication. The identity is synthesized before onboarding. The voice is cloned between help desk calls. The system verified correctly at the moment it checked. It just never checks again.
AI Augmentation
The defense against AI-enabled identity attacks does not require replacing your identity infrastructure. It requires adding a continuous verification layer on top of what you already have. The shift is from proving identity at a single point in time to continuously confirming that the entity using an account behaves like the person who was granted it.
Behavioral identity baselining from authentication logs. Your identity provider already generates the raw data for this. Every authentication event includes a timestamp, source IP, device identifier, user agent string, geographic location (derived from IP), and the resource being accessed. Over 30 to 60 days of collection, these events produce a behavioral fingerprint for each user: what time they typically log in, from which networks, using which devices, accessing which applications, in what sequence. The AI component is a lightweight anomaly detection model (isolation forest or autoencoder, either works for a small deployment) that scores each new authentication event against that user’s established baseline.
This is not impossible travel detection with a fresh coat of paint. Impossible travel checks a single variable (geographic distance divided by time) against a single threshold. Behavioral baselining checks a multidimensional pattern: this user logged in at 2 AM from a device they’ve never used, to an application they’ve never accessed, from a network block associated with a residential VPN provider, and their session immediately began enumerating file shares they’ve never touched. Each of those facts alone might be explainable. Together, they form a pattern that the model scores as anomalous, triggering a step-up authentication challenge or a hold for analyst review.
Export authentication events from your identity provider (Entra ID, Okta, Google Workspace) to a log store you can query. Write a baseline ingestion script that builds per-user profiles over the initial training window. Deploy an anomaly detection model that scores each new event and writes high-anomaly scores to your alerting pipeline. The model does not need to be a large deployment. An isolation forest running against authentication logs for 200 users can train on a single-core machine in minutes and score events in milliseconds.
The dependency risk is real: if the model is fed poisoned baseline data (an attacker who gains access and operates quietly during the training window), the model will learn the attacker’s behavior as normal. The manual fallback is threshold-based alerting on raw authentication variables with conservative thresholds, which is what you have today, supplemented by weekly manual review of accounts that accessed resources outside their normal application set.
AI-assisted help desk interaction analysis. This is the augmentation that addresses the Scattered Spider playbook directly. Record help desk calls (with appropriate notice and consent per your jurisdiction). Run the audio through a voice authentication model that compares the caller’s voice against a stored voiceprint for the claimed identity. But don’t stop at voiceprint matching, because the attacker’s cloned voice may beat the voiceprint comparison. Layer in analysis of the interaction itself.
The AI component analyzes the help desk call for patterns associated with social engineering: unusual urgency, specific request patterns (MFA reset is the number one target), emotional manipulation tactics, and discrepancies between the caller’s claimed identity and their speaking patterns. The system doesn’t make the access decision. It produces a risk score that the help desk agent sees on their screen during the call. A high-risk score triggers a callback verification to a known-good number, an out-of-band confirmation through the user’s registered device, or a hold for security team review. The goal is not to replace the agent’s judgment. The goal is to give the agent a signal when the social engineering is too subtle for a human to detect.
For a small team, the minimum viable version is simpler than it sounds. Implement a mandatory callback procedure for all MFA resets and privileged access changes, where the callback goes to the phone number on record in the HR system, not a number the caller provides. Layer in a voiceprint enrollment process for privileged users during their next in-person interaction or through a verified video call. Run transcribed help desk calls through a classification model trained on known social engineering patterns (datasets exist from security awareness training vendors). Alert when the transcript scores above a threshold for manipulation tactics.
The dependency risk is that the callback number in the HR system could itself be changed through a compromised HR portal or a social engineering attack against HR staff. The manual fallback is a physical verification requirement for high-privilege resets: the requesting employee must present in person or must be visually confirmed by their direct manager through an existing trusted channel.
Continuous session validation. This augmentation addresses the temporal gap by extending identity verification beyond the authentication moment. Once a user authenticates, their session is monitored for behavioral consistency with the authenticated identity. The data sources are application access logs, endpoint telemetry, and network flow data that you are probably already collecting.
The AI component builds a session-level behavioral model. During a normal session, this user accesses Outlook, Teams, and SharePoint in a particular pattern, generates a characteristic volume of network traffic, and interacts with a consistent set of internal resources. When a session begins accessing resources or generating traffic patterns that diverge from the authenticated user’s profile, the system can challenge the session with a transparent reauthentication prompt, step up monitoring, or terminate the session and require fresh authentication.
The practical implementation is a script that pulls session metadata from your identity provider and endpoint agent, correlates it against the user’s behavioral baseline (built from the same data used for authentication anomaly detection), and fires an alert or a reauthentication trigger when the session score crosses a threshold. The threshold should be tunable, because too aggressive a setting will lock out legitimate users who are doing something new and too lenient a setting will miss stolen sessions.
The dependency risk is that an attacker who steals a session and mimics the legitimate user’s behavior (accessing the same applications, generating similar traffic) will blend into the baseline. The manual fallback for that limitation is time-bounded session validity, where sessions expire and require reauthentication on a schedule regardless of behavioral scoring. Most identity providers already support session lifetime policies. Set them shorter than you are comfortable with and see what breaks.
Identity verification signal aggregation. This is the connective tissue that makes the other three augmentations work together. Instead of treating each identity signal (authentication event, help desk interaction, session behavior, access pattern) as an independent alert source, aggregate them into a per-identity risk score that updates continuously.
The AI component is a scoring engine that ingests signals from all three detection layers and produces a composite risk score per user identity per time window. An authentication anomaly alone might score a 3 out of 10. A help desk call with social engineering indicators might score a 4. But when the same identity shows an authentication anomaly at 2 AM, a help desk call requesting an MFA reset at 2:15 AM, and a new session accessing the domain controller at 2:30 AM, the composite score should be screaming. The aggregation layer catches attack chains that no individual detection layer would flag as critical on its own.
For a small team, this can start as a simple Python script that reads from the alert outputs of the other three components, applies weight-based scoring rules, and writes composite scores to a dashboard or alert channel. It does not need machine learning at the aggregation layer. Rule-based scoring with tuned weights based on your environment’s false positive rates will outperform a model that you don’t have enough training data to build.
The dependency risk for the aggregation layer is that it becomes a single point of failure: if the scoring engine goes down, all identity risk assessment stops. The manual fallback is the individual alert streams from each component, which continue to function independently. The aggregation layer adds correlation. It should not become a gate.
My family now has an entire verification-of-authenticity process just to make sure who is on the phone or at the other end of a text is the actual person. Challenge phrases. Callback procedures. Questions only we would know the answers to. We built that because we understood that the voice on the phone is no longer proof of identity. Your organization needs to internalize the same lesson, not as a policy statement, but as an operational reality built into every system that makes access decisions.
The waitress in Annapolis wasn’t careless. She was following the process she was given, and the process only checked whether a document existed, not whether it belonged to the person holding it. Your identity systems are doing the same thing. The question is whether you redesign the process before an attacker does it for you.
Works Cited
CSO Online. (2025, July 23). Clorox sues Cognizant for $380M over alleged helpdesk failures in cyberattack. https://www.csoonline.com/article/4027266/clorox-sues-cognizant-over-380m-over-alleged-helpdesk-failures-in-cyberattack.html
Cyber Monitoring Centre. (2025, June). M&S and Co-op cyber event assessment [Reported in Computing]. https://www.computing.co.uk/news/2025/security/cyber-monitoring-centre-estimates-cost-uk-retail-attacks
Dataminr. (2026, February 24). Cyber intel brief: Scattered Lapsus$ Hunters (SLH) kicks off campaign to recruit women. https://www.dataminr.com/resources/intel-brief/slh-recruiting-women-for-vishing/
Entrust. (2025, November 18). Entrust 2026 identity fraud report reveals surging attacks and diversifying tactics [Press release]. https://www.entrust.com/company/newsroom/deepfakes-social-engineering-and-injection-attacks-on-the-rise
Hitaj, B., Gasti, P., Ateniese, G., & Perez-Cruz, F. (2019). PassGAN: A deep learning approach for password guessing. arXiv. https://arxiv.org/abs/1709.00440
Hunt, T. (2025, November 5). 2 billion email addresses were exposed, and we indexed them all in Have I Been Pwned. https://www.troyhunt.com/2-billion-email-addresses-were-exposed-and-we-indexed-them-all-in-have-i-been-pwned/
IBM. (2025). Cost of a data breach report 2025. https://www.ibm.com/reports/data-breach
iProov. (2025, February 27). iProov issues annual identity verification threat intelligence report [Press release]. https://www.iproov.com/press/annual-identity-verification-threat-intelligence-report
KTVU/Associated Press. (2023, October). MGM Resorts estimates $100M loss from cyberattack that led to data breach. https://www.ktvu.com/news/mgm-resorts-cyberattack-data-breach-estimates-100m-loss
Mandiant. (2026). M-Trends 2026 [Reported in KnowBe4, “Voice phishing is a growing social engineering threat”]. https://blog.knowbe4.com/voice-phishing-is-a-growing-social-engineering-threat
Microsoft. (2025). Microsoft digital defense report 2025. https://www.microsoft.com/en-us/security/security-insider/threat-landscape/microsoft-digital-defense-report-2025
Palo Alto Networks Unit 42. (2025). 2025 Unit 42 global incident response report: Social engineering edition. https://unit42.paloaltonetworks.com/2025-unit-42-global-incident-response-report-social-engineering-edition/
Pindrop. (2025, June 12). New report: Deepfake fraud could surge 162% [2025 Voice Intelligence and Security Report]. https://www.pindrop.com/?p=46334
Shufti. (2026). Deepfake identity fraud index report 2026. https://shuftipro.com/resources/whitepapers-reports/deepfake-identity-fraud-index-report-2026/
SpyCloud. (2026, March 19). SpyCloud’s 2026 identity exposure report reveals explosion of non-human identity theft [Press release]. https://spycloud.com/newsroom/annual-identity-exposure-report-2026
Surfshark. (2026, July 14). $3.7B lost to deepfakes, social media is the primary origin. https://surfshark.com/research/deepfake-fraud-statistics
TransUnion. (2025). H1 2025 state of omnichannel fraud report. https://www.transunion.com/report/h1-2025-omnichannel-fraud-report
Verizon. (2025). 2025 data breach investigations report. https://www.verizon.com/business/resources/reports/dbir/
About the Author
