Google’s Threat Intelligence Group (GTIG) has published a detailed report describing how threat actors are systematically abusing the Gemini large language model (LLM) in real-world cyber operations. According to the findings, Gemini is being leveraged across the entire attack lifecycle — from reconnaissance and social engineering to infrastructure development and data exfiltration. A particularly concerning trend is the emergence of LLM distillation campaigns aimed at cloning Gemini’s capabilities at scale.
LLM Distillation at Scale: A New Risk to AI Intellectual Property
GTIG has observed multiple attempts to extract Gemini’s knowledge via mass automated querying. In one documented case, attackers sent the model more than 100,000 prompts in multiple languages, collecting the responses to train their own model intended to mimic Gemini’s behavior.
LLM distillation is a technique where a smaller or cheaper model is trained on the outputs of a more capable “teacher” model. Instead of investing billions of dollars and extensive compute resources in training from scratch, adversaries offload the most expensive research and development phases onto an existing commercial LLM.
This practice poses significant intellectual property and security risks. A distilled clone of a commercial model can:
— partially reproduce the decision logic and style of the original system;
— operate independently of the vendor’s safety controls and monitoring;
— be tuned specifically for generating malicious code, phishing content, or techniques for bypassing security tools without oversight from the original developer.
These concerns align with broader warnings from regulators and standards bodies that highlight model extraction and training-data leakage as critical AI security threats. For organizations investing in proprietary models, defending against automated distillation and abuse is becoming as important as traditional network and endpoint protection.
State-Sponsored APT Groups: Gemini in Support of Espionage and Exploitation
GTIG reports that state-aligned advanced persistent threat (APT) groups from China, Iran, North Korea, and Russia are actively experimenting with Gemini to enhance their cyber operations. The report references, among others, the Chinese group APT31 (Temp.HEX), the Iranian APT42, and North Korean cluster UNC2970.
Chinese Threat Actors: Vulnerability Analysis and Hexstrike MCP
Chinese operators were seen posing as information security professionals and using Gemini in seemingly “educational” scenarios. In reality, they sought assistance with automating vulnerability analysis and generating targeted test plans for specific systems, including Western targets.
GTIG highlights experimentation with the Hexstrike MCP toolset, where attackers asked Gemini to assist with:
— analyzing remote code execution (RCE) scenarios;
— exploring techniques for web application firewall (WAF) bypass;
— interpreting the results of SQL injection attacks against selected U.S. organizations.
Iran’s APT42: Social Engineering and Rapid Malware Development
The Iranian-linked group APT42 reportedly used Gemini as a rapid development platform for malicious tooling. GTIG notes that the model was employed to:
— draft convincing phishing emails and social engineering scripts tailored to specific victims;
— generate and debug code snippets used in offensive tooling;
— study exploitation techniques and integrate them into custom malware frameworks.
GTIG further indicates that North Korean and Russian entities rely on Gemini to streamline target reconnaissance, design command-and-control (C2) infrastructures, and optimize data theft workflows, underscoring that generative AI is now an operational asset for multiple APT ecosystems.
AI-Driven Malware: CoinBait and HonestCue as Emerging Case Studies
CoinBait: AI-Assisted Phishing Kit Masquerading as a Crypto Exchange
The report describes CoinBait, a phishing kit implemented as a single-page application built on React. The kit impersonates a legitimate cryptocurrency exchange with the goal of harvesting user credentials and other sensitive data.
Code analysis revealed artifacts characteristic of generative AI–assisted development. In particular, log messages prefixed with “Analytics:” suggest automated code generation patterns. Additional indicators point to the use of the Lovable AI platform, including references to the Lovable Supabase client and the lovable.app domain within the project.
HonestCue: Gemini-Integrated Loader Generating Second-Stage Payloads
Another example, HonestCue, is described as a proof-of-concept loader framework integrated with the Gemini API. Its purpose is to dynamically generate and execute second-stage malware components.
The operation flow of HonestCue includes:
— calling the Gemini API to generate C# code for the second-stage module on demand;
— compiling the generated code in memory;
— executing the resulting payload without writing it to disk, which significantly complicates detection by traditional antivirus and many endpoint detection and response (EDR) products.
Google’s Defensive Measures and Key Takeaways for Security Teams
Google emphasizes that all identified malicious accounts and associated infrastructure have been suspended or blocked. The company has also strengthened Gemini’s resilience by updating classifiers, deploying stricter prompt and response filtering policies, and enhancing systems for detecting anomalous activity, including large-scale distillation attempts.
The abuse of Gemini illustrates a broader trend: generative AI is a dual-use technology. It can augment defenders with better detection, response, and automation — but it equally amplifies attackers by lowering the barrier to entry and accelerating every step of the kill chain.
Organizations should proactively adapt their security posture by:
— updating threat models to include LLM misuse, model extraction, and API abuse;
— defining and enforcing corporate policies for safe use of AI services (internal and external);
— monitoring for anomalous LLM API usage patterns, such as mass prompt generation or suspicious code requests;
— continuously training staff to recognize AI-enhanced phishing and social engineering campaigns.
As large language models become embedded in everyday business processes, their security, resilience against distillation, and protection from abuse must be treated on par with traditional cybersecurity controls. Investing in AI-specific defenses today — from access control and logging to abuse detection and IP protection — will directly influence an organization’s resilience against the next generation of AI-powered cyber threats.
