AI chatbots are giving out people’s real phone numbers

People report that their personal contact info was surfaced by Google AI—and there’s apparently no easy way to prevent it.

A Redditor recently wrote that he was “desperate for help”: for about a month, he said, his phone had been inundated by calls from “strangers” who were “looking for a lawyer, a product designer, a locksmith.” Callers were apparently misdirected by Google’s generative AI.

In March, a software developer in Israel was contacted on WhatsApp after Google’s chatbot Gemini provided incorrect customer service instructions that included his number.

And in April, a PhD candidate at the University of Washington was messing around on Gemini and got it to cough up her colleague’s personal cell phone number.

AI researchers and online privacy experts have long warned of the myriad dangers generative AI poses for personal privacy. These cases give us yet another scenario to worry about: generative AI exposing people’s real phone numbers.

A 400% increase in AI-related privacy requests

It’s impossible to know how often people’s phone numbers are exposed by AI chatbots, but experts say they believe that it is happening far more than is reported publicly.

DeleteMe, a company that helps customers remove their personal information from the internet, says customer queries about generative AI have increased by 400%—up to a few thousand—in the last seven months. These queries “specifically reference ChatGPT, Claude, Gemini … or other generative AI tools,” says Rob Shavell, the company’s cofounder and CEO. Specifically, 55% of these concerns about generative AI reference ChatGPT, 20% reference Gemini, 15% Claude, and 10% other AI tools, Shavell says.

Shavell says customer complaints about personal information being surfaced by LLMs usually take two forms: Either “a customer asks a chatbot something innocuous about themselves and gets back accurate home addresses, phone numbers, family members’ names, or employer details.” Alternatively, a customer may be confronted with and report the exposure of someone else’s personal data, when “the chatbot generates plausible-but-wrong contact information.”

This aligns with what happened to Daniel Abraham, a 28-year-old software engineer in Israel. In mid-March, he says, a stranger sent him a “weird WhatsApp message from an unknown number” asking for help with his account in PayBox, an Israeli payment app. When he asked the stranger how they had found his number, they sent him a screenshot of Gemini’s instructions to contact PayBox customer service via WhatsApp—giving his personal number. Abraham does not work for PayBox, and PayBox does not have a WhatsApp customer service number.

The Training Data Problem

Chatbots like Gemini, OpenAI’s ChatGPT, and Anthropic’s Claude are built on LLMs that are trained on huge amounts of data scraped from across the web. This inevitably includes hundreds of millions of instances of PII. The likelihood of PII appearing in AI training data is only increasing as public data “runs out” and AI companies look for new sources of high-quality training data, including information from data brokers. According to the California data broker registry, 31 of 578 registered data brokers self-reported that they had “shared or sold consumers’ data to a developer of a GenAI system or model in the past year.”

Imperfect Measures

It’s standard practice now to build guardrails into an LLM’s design to constrain certain outputs, but these safeguards don’t always work. Meira Gilbert, a PhD student at the University of Washington, searched for her friend's contact info on Gemini. After providing a research overview, Gemini returned the friend’s personal phone number.

In another test with ChatGPT regarding a professor's info, OpenAI’s guardrails initially kicked in. However, the chatbot then suggested an “investigative-style” approach to “narrow things down.” By following the chatbot's own suggestions to provide minor clues, the students were able to get ChatGPT to produce the professor’s home address, purchase price, and spouse’s name from city records.