Scammers Used Google's Own AI to Fake a Million Sites

Google filed suit on Thursday against Outsider Enterprise, a China-based cybercrime network that, according to the complaint, turned Google's own Gemini AI into a phishing factory. The group ran a subscription business — roughly $88 a week or $200 a month — selling software that used Gemini to mass-produce fake websites. It shipped with more than 290 ready-made templates impersonating phone carriers, banks, government services, and retailers, plus AI-generated code guides so a buyer barely had to know what they were doing.

The numbers are the part that should make you sit up. In a single two-week window in May, the operation blasted out 2.5 million scam text messages; Android users flagged 55,000 of those as spam in that same span. Google counted around 9,000 live fake sites, close to a million fraudulent domains, and 1.59 million malicious URLs over five months between November and April. The haul: more than 36,000 stolen payment cards from institutions across 95 countries, hundreds of thousands of victims, and losses running into the millions. For context, the FBI ties this style of attack to 3.87 million stolen cards and $1.9 billion in losses since mid-2023.

Google's lawsuit alleges trademark and copyright infringement, racketeering, wire fraud, and false advertising, and asks the court for damages plus an injunction to shut the operation down. It didn't do this alone: the takedown was coordinated with AT&T, T-Mobile, Verizon, the FBI, and Lumen's Black Lotus Labs to block messages and seize domains and the Shopify storefronts the group used to cash out.

Why this lands in your texts, not someone else's

I build with AI every single day, and I'll be honest — this is the part of the AI story I don't enjoy writing. The exact capability that lets me spin up a working web page in an afternoon is what lets a scammer spin up ten thousand fake ones overnight. For years, the fake text gave itself away: clumsy grammar, a wrong logo, a link that looked off. AI quietly fixed all three. The fake bank page now looks like your bank's page because a model copied it pixel for pixel, and the message reads like a real human wrote it because, functionally, a very capable one did.

So the advice I grew up on — "watch for typos" — is dead. The thing that still works is boring and bulletproof: judge the channel, not the content. A real bank, a real delivery company, a real tax office will never need you to follow a link from an unexpected text to verify your account or pay a surprise fee. The message can be flawless and still be a trap. The tell isn't a misspelling anymore; it's the combination of urgency plus a link that pulls you off the app you trust and onto a page you've never seen.

Here's the uncomfortable takeaway from someone who teaches people to build with AI: the tools are neutral, and neutral cuts both ways. The same week I show a student how to clone a landing page as a harmless exercise, someone on the other side of the world is cloning your bank's page for profit. Understanding how the trick is assembled is now part of basic self-defense — which is exactly why I'd rather you understand this than be afraid of it. Fear makes you click; understanding makes you slow down.