Translated from 日本語 • View original 
GTIG Warns of "Thinking Malware" :P ROMPT FLUX and PROMPTSTEAL
GTIG has discovered PROMPTFLUX and PROMPTSTEAL as emerging malware types that exploit LLMs (large language models).
They bypass traditional security detections by having the AI "dynamically generate" or "self-obfuscate code" rather than writing code.
1. PROMPTFLUX: A Dropper That Self-Mutuates in Gemini
PROMPTFLUX is a dropper created with VBScript, but its biggest feature is that it uses Google's generative AI "Gemini" to obfuscate and regenerate its own code.
The malware has a built-in module called "Thinking Robot" that periodically sends queries to the Gemini API to generate new code to bypass antivirus products.
- How it works: Use a hard-coded API key to send a POST request to Gemini's endpoint.
- Persistence: Always specify and call the stable "gemini-1.5-flash-latest" model to increase the resiliency of the tool.
- Purpose: More than just obfuscation, it rewrites and stores the code itself after infection, constantly continuing to operate as a new variant.
2. PROMPTSTEAL: A data miner that lets AI create instructions
On the other hand, PROMPTSTEAL is a malware used by the Russian government-backed hacker group APT28 to attack Ukraine.
The malware does not have specific stealing commands inside. Instead, it instructs LLM APIs such as Hugging Face to "create a command to collect system information" and executes the AI-generated command.
- Impersonation: Send malicious queries in the background while pretending to be an image generator to the user.
- Prompt example: "List commands to recursively copy PDFs and txts from the user's documents folder."
The Reality of AI Exploitation by State-Sponsored Hackers
The GTIG report also details how certain state-sponsored threat actors are abusing AI like Gemini for their own purposes.
China: Vulnerability Exploration Posing as a CTF Participant
Chinese threat actors use Gemini under the guise of a Capture The Flag (CTF) participant.
They ask AI to "tell me the vulnerability of this system" and use the information obtained to develop exploits, web shells, and scout cloud infrastructure (such as Kubernetes). Organizations such as APT41 are also using AI to obfuscate the code of the C2 framework OSSTUN.
North Korea (UNC1069): Social Engineering for Crypto Assets
The North Korea-related organization "UNC1069" is abusing Gemini to steal cryptocurrency (virtual currency).
- Research to locate your wallet app's data.
- Generate plausible business communication and meeting change requests in Spanish.
- Disguise to disguise software updates.
Iran (TEMP. Zagros / APT42): Impersonation and deepfake of students
The Iranian organization "TEMP. Zagros tries to get students to write malicious scripts that they claim are "necessary for university projects" to circumvent AI safeguards (such as ethical filters).
APT42 also uses deepfake images and videos to deceive relevant parties into distributing the backdoor "BIGMACHO".
Threat Point: AI Proliferation in the Underground
In addition to these sophisticated attacks, underground forums also widely distribute AI tools dedicated to cybercrime, such as WormGPT and FraudGPT.
They offer capabilities that can be exploited by attackers of any skill level, from creating phishing emails to developing malware and exploiting vulnerabilities.
Countermeasures: Countermeasures against AI Abuse
As long as attackers use AI, defenders also need to take measures with AI in mind. The report recommends the following response guides:
- Monitor and block AI API communication
- Monitor API calls from your internal network to unnecessary LLM services (OpenAI, Gemini, Hugging Face, etc.) and block non-business-related communications.
- Pattern detection of AI-generated commands
- Incorporate unique AI-generated command execution patterns and unnatural script behavior into detection rules (SIEM/EDR), such as PROMPTSTEAL.
- Token protection for development environments
- Attackers target hardcoded API keys, etc. It is important to strictly manage authentication tokens and API keys in the development environment to prevent leakage.
Conclusion: Cat-and-mouse is moving towards the era of "AI vs. AI"
The GTIG report shows that malware has evolved from "static code" to "dynamically thinking and changing threats".
Like PROMPTFLUX, the era has arrived when malware itself consults AI, "How can I not find it in antivirus software?" and mutates itself.
We defenders also need to keep an eye out for AI-specific communication patterns and behaviors while adhering to the basic principles (defense-in-depth, least privilege).
That's it for Yoon Jae-ho of Colorkrew Security.
Thank you for reading to the end.