How Large Language Models Are Breaking Online Anonymity

CyberSecureFox 🦊

For years, many users have assumed that a pseudonym on social media is enough to stay anonymous. Recent research from ETH Zurich, the MATS (ML Alignment & Theory Scholars) program and Anthropic shows that this assumption is rapidly becoming obsolete. The study demonstrates that modern large language models (LLMs) can systematically deanonymize user accounts using only publicly available data.

LLM deanonymization: matching anonymous accounts to real identities

The researchers evaluated how effectively LLMs can perform deanonymization—linking anonymous profiles to specific individuals based on their online activity. In several experiments, the models achieved up to 68% recall (share of all targeted accounts that were correctly deanonymized) and up to 90% precision (share of correct matches among all predicted identities).

Traditional deanonymization methods rely on manually curated, structured datasets and often require significant expert effort. Their accuracy tends to be modest and difficult to scale. In contrast, LLMs process unstructured text at scale and can extract weak, nuanced signals—such as wording, interests and context—that were previously hard to exploit automatically.

Hacker News and LinkedIn: recovering links after identifiers are removed

In one experiment, the team used public posts from Hacker News and corresponding LinkedIn profiles, matched initially via explicit cross-platform mentions. All direct identifiers—names, usernames, URLs—were then stripped from the data. The LLM was tasked with reconstructing which anonymized Hacker News accounts belonged to which LinkedIn users based solely on the content of their contributions.

The model leveraged technical topics, writing style, self-described roles and career histories embedded in the text to match accounts. This scenario closely mirrors realistic risks: even if obvious identifiers are removed or obfuscated, the content itself can be enough to reveal identity.

Netflix-style viewing data: scaling classic deanonymization attacks

Another dataset was built from historical Netflix usage information—movie preferences, recommendations, viewing records and transactions. This revisits the well-known Netflix Prize incident, where researchers in the late 2000s showed that supposedly anonymized movie ratings could be linked to individuals using external sources like IMDb.

The new work suggests that LLM deanonymization can now automate and scale similar attacks. Instead of manually crafting statistical models for specific datasets, attackers could prompt a general-purpose LLM to correlate patterns of behavior, timing and content across multiple platforms.

Reddit stylometry: linking fragments of activity to one user

A separate experiment focused on Reddit. Researchers split users’ comment histories into fragments and asked the LLM to decide whether two pieces of text were written by the same person based on writing style and interests—a form of AI-driven stylometry.

The results were particularly striking in movie-related communities: r/movies and five related subreddits (r/horror, r/MovieSuggestions, r/Letterboxd, r/TrueFilm, r/MovieDetails). At a precision of 90%, the models were able to deanonymize:

3.1% of users who discussed a single film, 8.4% of users who discussed 5–9 films, and 48.1% of users who wrote about 10 or more movies. In other words, ordinary engagement with film discussions can form a distinctive behavioral and linguistic fingerprint that is difficult to hide.

Free-text surveys: identifying real people from AI usage descriptions

In another test, researchers analyzed answers from 125 participants in an Anthropic survey about everyday AI usage. Based only on these free-text responses, the LLM could correctly identify about 7% of respondents by linking them to external information. While this percentage may seem small, it is notable because the source text was unconstrained and not obviously identifying.

Why large language models amplify online anonymity risks

To compare LLMs with classic techniques, the authors recreated a Netflix Prize–style attack using data from 5,000 Reddit users plus 5,000 synthetic profiles. As the matching task became more complex, traditional statistical methods quickly lost accuracy, whereas the LLM maintained more stable and reliable performance.

The key difference is that LLMs model more than explicit attributes such as timestamps or overlapping movie titles. They capture semantics and style—vocabulary, sentence structure, typical themes and combinations of interests. Together, these features form a linguistic “fingerprint” that can identify a person even when names and usernames are hidden. Earlier stylometry research showed this effect on a small scale; LLMs make it faster, cheaper and far more scalable.

Implications for users, businesses and governments

If the accuracy and scale of LLM-powered deanonymization continue to improve, the impact on online anonymity could be profound. Governments could, in principle, use such tools to unmask anonymous critics, whistleblowers or opposition voices. Corporations might build ultra-granular behavioral and advertising profiles that go far beyond what users expect or consent to.

For cybercriminals, easier deanonymization lowers the barrier to targeted attacks. Once pseudonymous accounts are linked to real people, attackers can combine public posts, leaked databases and social media to craft convincing social engineering, phishing, extortion and doxing campaigns. The danger is compounded by a widespread but outdated belief that multiple nicknames and separated accounts provide sufficient protection.

Mitigation strategies: responsibilities for platforms, LLM providers and users

How platforms can reduce deanonymization risks

Online services should limit the rate and volume of API access to user-generated content, deploy robust anti-scraping mechanisms and monitor for large-scale data harvesting. Restricting bulk export of posts and interaction histories makes it harder to build the massive corpora that LLM deanonymization thrives on.

Responsible use by LLM providers

Vendors of large language models can embed usage policies that explicitly ban deanonymization and profiling of individuals. At the technical level, providers can analyze patterns of queries and block scenarios where models are systematically used to link accounts or infer hidden personal data. Enforcement at the infrastructure layer is essential; relying only on end-user ethics is insufficient.

Practical steps for individual users

Users should revisit their personal threat models. Recommended practices include avoiding reuse of the same handle, bio and links across platforms; minimizing unnecessary personal details in public posts; segregating professional and personal activity into distinct accounts; and regularly auditing privacy settings. For those who require stronger anonymity—such as activists or journalists—additional measures like Tor, compartmentalized identities and strict operational security are crucial.

The evolution of large language models shows that pseudonyms and basic anonymization are no longer robust shields. Text itself—what we write, how we phrase it and which topics we return to—has become an identifier. As LLM deanonymization matures, preserving control over digital identity will depend on a holistic approach to managing one’s digital footprint. The earlier users, organizations and regulators adapt security and privacy practices to this new reality, the better positioned they will be to defend online anonymity in an AI-driven world.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.