The OWASP Foundation, known for its "Top 10" lists of critical web application security risks, has extended its focus to the rapidly evolving landscape of Large Language Models (LLMs) and Generative AI. They have developed the OWASP Top 10 for LLM Applications (and more broadly, the OWASP Gen AI Security Project) to address the unique security challenges posed by these technologies.
Here are the key security risks identified by OWASP for LLM and GenAI projects (as per the 2025 updates):
OWASP Top 10 for LLM Applications (2025)
LLM01: Prompt Injection: This is the most critical risk. Attackers manipulate the LLM's input prompts to alter its behavior, potentially causing it to generate misleading, harmful, or unauthorized outputs, or even to perform actions beyond its intended scope. This can be direct (overwriting system prompts) or indirect (injecting malicious data into external sources the LLM processes).
LLM02: Sensitive Information Disclosure: LLMs can unintentionally reveal confidential or private data through their responses or outputs. This could include PII (Personally Identifiable Information), proprietary algorithms, or other sensitive business data.
LLM03: Supply Chain Vulnerabilities: Risks associated with third-party dependencies and components used in developing or deploying LLMs (e.g., pre-trained models, datasets, libraries) can introduce security weaknesses and potential backdoors.
LLM04: Data and Model Poisoning: Adversaries intentionally introduce incorrect or biased data into a machine learning model's training set to manipulate the model's outcomes, degrade its performance, or introduce vulnerabilities.
LLM05: Improper Output Handling: Failure to properly manage and sanitize the outputs generated by an LLM before they are passed downstream to other systems. This can lead to injection attacks (like XSS or SQL injection) or other security vulnerabilities.
LLM06: Excessive Agency: This risk arises when an LLM is given too much autonomy or control, potentially leading it to execute harmful actions or make decisions beyond its intended scope (e.g., allowing an LLM to write or delete files when it should only read).
LLM07: System Prompt Leakage: Sensitive information within the system prompts (instructions guiding the LLM's behavior) is exposed. This can reveal secrets or internal logic that an attacker could exploit.
LLM08: Vector and Embedding Weaknesses: This new entry focuses on the security of Retrieval-Augmented Generation (RAG) and embedding-based methods. Vulnerabilities include malicious data injections, embedding poisoning, unauthorized access, and cross-context information leaks.
LLM09: Misinformation: LLMs can produce credible-sounding yet false or biased content (often due to hallucinations or biases in training data). Overreliance on these outputs without verification can lead to critical errors, reputational damage, or legal liabilities.
LLM10: Unbounded Consumption: Attacks that disrupt or exhaust an LLM's resources, rendering it unable to process legitimate requests or operate effectively. This can lead to Denial of Service (DoS) attacks, increased operational costs, or even model theft.
Other Important Aspects of the OWASP Gen AI Security Project:
Beyond the Top 10 list, the OWASP Gen AI Security Project is a broader initiative that also includes:
OWASP GenAI Red Teaming Guide: A guide for comprehensively testing generative AI systems to identify vulnerabilities.
LLM Security Verification Standard: A checklist to help design and test LLM-based applications.
AI Security Solutions Landscape Guide: Cataloging emerging AI security solutions.
Agentic AI Security Initiative: Dedicated to securing autonomous AI systems and agent-based LLM applications.
LLM & Generative AI Data Security Best Practices Guide: Covers best practices for protecting privacy and mitigating threats related to data in GenAI.
Why is this important for LLM and GenAI projects?
As organizations rapidly adopt LLMs and generative AI, understanding and mitigating these specific security risks is crucial. The OWASP Gen AI Security Project provides a vital framework and resources for:
Identifying and documenting critical risks: Helping developers and security professionals understand the unique attack vectors.
Developing actionable mitigation strategies: Providing guidance on how to secure these systems throughout their lifecycle.
Promoting secure and responsible AI deployment: Fostering a more secure ecosystem for AI technologies.
By incorporating the principles and guidelines from OWASP for LLM and GenAI, development teams can build more resilient, trustworthy, and secure AI-driven applications.
