Written by Laurent Dupont

Mind the Gap: STRIDE-AI

Your Clear Path to Understanding AI Vulnerabilities

The security landscape is changing rapidly as AI integrates into every part of our lives – from smart assistants and recommendation systems to autonomous vehicles and vision technology. While traditional cybersecurity practices remain essential, AI-enabled systems introduce new types of threats that require a specialized approach. At Toreon, we’ve experienced this firsthand. That’s why we’re excited to launch STRIDE-AI, our enhanced methodology for comprehensive AI threat modeling, along with our new 3-day AI threat modeling training.

Why Traditional STRIDE Isn't Enough for AI

The STRIDE threat modeling framework, originally created by Microsoft, has been fundamental to application security for many years. It classifies threats into six categories: Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege. These categories remain relevant for AI systems. However, how these threats appear in an AI setting is significantly different, requiring a deeper understanding and customized countermeasures. AI systems aren’t just code; they learn, infer, and adapt, making them susceptible to unique attack vectors like data poisoning, adversarial examples, and prompt injection. This is where STRIDE-AI comes in, extending the classic framework to address the complexities of AI-specific risks.

Let’s examine how each STRIDE category applies, using real-world examples to highlight the urgency of this specialized approach.

STRIDE-AI: A Deep Dive into AI-Specific Threats

1. Spoofing: Impersonation in the Age of AI

Manifestation: Attackers impersonate users, AI services, or data sources to fool an AI system. Think deepfakes bypassing biometric authentication or forged sensor feeds misleading an autonomous system.
Real-World Example: The infamous case of criminals using AI to clone a CEO’s voice, authorizing fraudulent transactions worth hundreds of thousands of dollars. Another chilling example: a small piece of tape on a speed-limit sign caused Tesla’s image recognition to misread “35” as “85,” leading the car to accelerate dangerously. These are stark illustrations of AI spoofing – forged inputs accepted as genuine.
Our Approach: We emphasize strong authentication and validation. This goes beyond simple passwords, advocating for multi-factor authentication, cross-modal checks (like liveness tests for biometrics), and cryptographic authentication of AI model endpoints and data sources. Deepfake detection and anomaly detection are also crucial.

2. Tampering: Corrupting AI's Integrity

Manifestation: Unauthorized alteration of data or models in the AI pipeline. This includes data poisoning (injecting malicious data into training sets), model poisoning/backdooring (altering model weights), or adversarial input attacks (subtly manipulating inputs to cause incorrect outputs).
Real-World Example: In 2016, Microsoft’s Tay Twitter bot was quickly “poisoned” by users feeding it hateful inputs, forcing its shutdown. Similarly, placing tape on a road sign (tampering with the physical input) fooled a Tesla’s vision system. These incidents underscore how even subtle malicious modifications can compromise an AI’s integrity.
Our Approach: We focus on robust data and model integrity controls. This involves securing the entire AI training pipeline with trusted data sources, versioning, and validation (e.g., outlier detection for poisoned data). We also cover robust and adversarial training techniques and protect model files with encryption and digital signatures. Continuous monitoring of model performance is key to detecting tampering.

3. Repudiation: The Challenge of Accountability in AI

Manifestation: The lack of accountability or traceability in AI operations allows parties to deny actions. This occurs when insufficient logs or audit trails exist for model training, data access, or decision outputs.
Real-World Example: A deepfake audio clip used in a UK court to falsely incriminate someone, forcing the person to repudiate it – a clear demonstration of how AI can complicate accountability. Without proper logging, an autonomous agent making an unauthorized transaction could be easily denied, with “the AI” taking the blame.
Our Approach: We emphasize comprehensive AI audit logs that are tamper-evident, ensuring an immutable record. Digital signatures or watermarks on critical model outputs can prove authenticity, and clear accountability for AI actions is established by associating them with user or process IDs

4. Information Disclosure: AI as a Data Leakage Vector

Manifestation: Unauthorized access or exposure of sensitive data through AI systems (privacy leakage). Model reverse engineering contributes significantly here, enabling attackers to reconstruct sensitive training data or infer confidential insights from an AI model, essentially turning the AI into an unintentional data leakage vector. Attackers can exploit models to extract training data (model inversion, membership inference), or an AI system might inadvertently reveal confidential information in its responses.
Real-World Example: Researchers demonstrated how simple prompts could trick ChatGPT into revealing over 10,000 pieces of verbatim training data, including names, phone numbers, and addresses. Another instance involved attackers using membership inference to determine if a specific user’s data was part of a model’s training set, despite it being private.
Our Approach: We champion privacy-preserving AI techniques like differential privacy during training to reduce memorization. Strict access controls, output filters to redact sensitive data, and secure handling of trained models (treating them as sensitive assets) are also crucial to prevent information leakage.

5. Denial of Service (DoS): Crippling AI Availability

Manifestation: Disrupting an AI system or reducing its performance by overwhelming resources or exploiting worst-case behaviors. This can involve flooding an AI service with requests, creating expensive inputs, or even data poisoning aimed at availability.
Real-World Example: Sending prompts to Large Language Models (LLMs) that trigger extremely large outputs or infinite loops, leading to service slowdowns or crashes. The OWASP Top 10 for LLMs notes that overloading LLMs with resource-intensive tasks can disrupt service and significantly increase costs. For example, a single carefully crafted query to an LLM application has been shown to potentially result in a bill exceeding $1000 due to excessive token generation and processing.
Our Approach: We advocate for robust measures like rate limiting and input throttling on AI service endpoints. Input validation to reject unusually large or complex inputs, performance monitoring, and graceful degradation strategies are key to maintaining availability under attack.

6. Elevation of Privilege: Gaining Unintended Control via AI

Manifestation: Gaining higher privileges or capabilities via the AI system than intended. This can involve exploiting vulnerabilities in AI plugins to execute code on the host, or “jailbreaking” an AI’s safeguards to make it perform restricted actions.
Real-World Example: The “DAN” (Do Anything Now) exploit in ChatGPT, where users crafted prompts to bypass the model’s safety rules and generate disallowed content. Another example is an insecure AI plugin that allows malicious prompts to inject commands for execution on the server, leading to remote code execution (RCE).
Our Approach: We emphasize strict policy enforcement, segmentation, and sandboxing for AI systems. AI agents and plugins should operate with the least privileges possible within isolated environments. Robust input validation and prompt filtering are essential to neutralize malicious patterns and prevent the AI from becoming an attack vector for higher system access.

Beyond STRIDE-AI: A Holistic View

While STRIDE-AI forms the core of our methodology, we also integrate insights from other crucial frameworks:

NIST AI Risk Management Framework (AI RMF 1.0) provides comprehensive guidance for managing AI system risks throughout their lifecycle.
MITRE ATLAS (Adversarial Threat Landscape for AI): A knowledge base of adversary tactics and techniques specifically targeting AI systems.
OWASP Machine Learning Security Top 10 (2023) & OWASP Top 10 for LLM Applications (2024): Industry-focused checklists outlining common vulnerabilities and attack vectors in ML and LLM systems.
AI Incident Database (AIID): A valuable resource for understanding real-world AI failures and incidents, grounding our threat modeling in practical examples.

It’s crucial to remember that AI integration doesn’t negate the need for traditional application security (appsec). Since AI systems are often embedded within larger technological infrastructures, robust conventional security measures remain vital to protect the entire ecosystem.

Strengthen Your AI Security with Toreon's Expertise

Understanding these AI-specific threats is now essential. As organizations increasingly adopt AI, proactive and specialized security measures become crucial. At Toreon, we’ve created a 3-day AI Threat Modeling Training that provides security and engineering professionals with practical skills and a structured approach to identify, assess, and mitigate threats in AI applications.

Using our enhanced STRIDE-AI methodology, you’ll learn to systematically build robust security into your AI systems from the ground up. The training culminates in an engaging Red Team/Blue Team wargame, allowing you to put your newfound skills to the test in a realistic scenario.

Don’t let AI’s unique risks catch you off guard. Invest in your organization’s security in the future.

Ready to master AI threat modeling?

About the Author

Seba Deleersnyder is the editor of the Threat Modeling Insider newsletter and a passionate advocate for practical security solutions. With years of experience in the field, he continues to curate insights and build communities that make threat modeling more accessible to everyone.