Rogue Agent Watch › Records › microsoft-openai-state-actor-llm
Microsoft/OpenAI disruption of state-affiliated actors misusing LLMs (2024)
Disclosed · added to the index · last updated
Grades
- Verification status
- Confirmed — Confirmed by a first-party disclosure or multiple independent credible sources.
- Sourcing confidence
- Primary — Backed by primary sourcing — first-party disclosure, vendor incident report, government advisory, or court documents.
- AI role
- Incidental — AI played a minor or supporting role (e.g. a productivity aid); sources indicate it did not provide novel capability.
- Severity
- Medium — Limited or contained harm, or high-signal capability demonstration.
Summary
On 2024-02-14 Microsoft Threat Intelligence and OpenAI jointly disclosed that they had detected and disrupted five state-affiliated threat actors using OpenAI's large language models to support cyber operations: Forest Blizzard (Russia), Emerald Sleet (North Korea), Crimson Sandstorm (Iran) and the China-affiliated Charcoal Typhoon and Salmon Typhoon. Reported uses included reconnaissance, scripting help, vulnerability research and social-engineering content. Both companies stated the activity amounted to productivity support rather than novel AI-enabled attack techniques, and OpenAI terminated the associated accounts.
Impact as stated
No novel or unique AI-enabled attack techniques were observed; Microsoft and OpenAI characterised the activity as consistent with using AI as a productivity tool. Identified accounts were terminated.
Facts as stated by sources
- Actor
- Five state-affiliated actors: Forest Blizzard, Emerald Sleet, Crimson Sandstorm, Charcoal Typhoon, Salmon Typhoon (Nation-state)
- Category
- Infrastructure abuse / supply chain
- Models named
- GPT-4
- Model families
- GPT (OpenAI)
- Agentic autonomy level
- Tool-assisted
- Guardrail bypass
- Legitimate tool abuse
- Attack lifecycle phases
- Reconnaissance, Resource development, Deception / social engineering
- Target sectors
- not stated
- Target countries
- not stated
- Organisations affected
- not stated
- Records exfiltrated
- not stated
Framework mappings
- MITRE ATLAS
- AML.T0016.002
Mitigations as stated
- The provider terminated the accounts associated with the identified actors.
Map points
- origin: Russia (state sponsor of Forest Blizzard, per Microsoft) (illustrative, country-level centroid; sponsor attribution, per Microsoft Threat Intelligence; RU)
- origin: North Korea (state sponsor of Emerald Sleet, per Microsoft) (illustrative, country-level centroid; sponsor attribution, per Microsoft Threat Intelligence; KP)
- origin: Iran (state sponsor of Crimson Sandstorm, per Microsoft) (illustrative, country-level centroid; sponsor attribution, per Microsoft Threat Intelligence; IR)
- origin: China (state sponsor of Charcoal Typhoon and Salmon Typhoon, per Microsoft) (illustrative, country-level centroid; sponsor attribution, per Microsoft Threat Intelligence; CN)
Sources (3)
- Staying ahead of threat actors in the age of AI
Microsoft Threat Intelligence · First-party disclosure · · archived copy - OpenAI, Microsoft warn of state-linked actors' AI use
Cybersecurity Dive · News · · archived copy - Microsoft, OpenAI reveal ChatGPT use by state-sponsored hackers
SC Media · News · · archived copy
Cite this record
Agentic Attack Index (MLSecOpsHub), dataset v0.3.0, record "microsoft-openai-state-actor-llm". https://raw.githubusercontent.com/MLSecOpsHub/agentic-attack-index/main/dist/incidents/microsoft-openai-state-actor-llm.json — CC BY-SA 4.0.