OWASP Top 10 for Large Language Model Applications

The OWASP Top 10 List for Large Language Models version 0.1 is a draft of the most significant vulnerability types in Artificial Intelligence (AI) applications built on Large Language Models (LLMs).

LLM01:2023 - Prompt Injections

Prompt injections are vulnerabilities that can lead to harmful consequences such as data leakage, unauthorized access, or other security breaches. To prevent and mitigate these risks, developers should implement prompt input validation and sanitization while regularly updating and fine-tuning the LLM to improve its ability to recognize malicious inputs and edge cases. Additionally, monitoring and logging LLM interactions helps detect and analyze potential prompt injection attempts early on.

LLM02:2023 - Data Leakage

Data leakage occurs when an LLM unintentionally exposes confidential or sensitive information. The associated risks can be minimized through several measures: implementing output filtering and context-aware mechanisms, applying data anonymization and differential privacy techniques during training, and conducting regular audits and logging of LLM interactions. Through vigilant monitoring and strict security protocols, developers can significantly reduce the risk of data leakage and ensure the secure operation of their LLMs.

LLM03:2023 - Inadequate Sandboxing

Sandboxing is a security technique used to limit an LLM's access to external resources or sensitive systems. When sandboxing is inadequately implemented, it can enable exploitation, unauthorized access, or unintended actions by the LLM. To mitigate these risks, proper sandboxing should isolate the LLM environment from critical systems and resources while restricting the LLM's capabilities and access to only what is necessary. Developers should also regularly audit the LLM environment, access controls, and interactions to detect potential sandboxing issues. Typical attack scenarios include exploiting an LLM's access to a sensitive database for confidential information or manipulating the LLM to execute unauthorized commands.

LLM04:2023 - Unauthorized Code Execution

Unauthorized code execution is a vulnerability where an attacker subverts an LLM to execute malicious code, commands, or actions on the underlying system. Countermeasures include stringent input validation and sanitization, proper sandboxing, restricting the LLM's capabilities, regular auditing of the LLM environment and access controls, and monitoring and logging of interactions. Typical attack scenarios include crafting a prompt to launch a reverse shell on the underlying system or manipulating the LLM into executing unauthorized system actions. By understanding these risks and taking appropriate steps, developers can safeguard their LLMs and protect their systems.

LLM05:2023 - SSRF Vulnerabilities

Server-side Request Forgery (SSRF) vulnerabilities arise from inadequate input validation, insufficient sandboxing and resource restrictions, or misconfigured network and application security settings. To prevent such incidents, organizations should implement rigorous input validation, sandboxing and resource restrictions, network and application security reviews, and monitoring and logging of LLM interactions. Attackers frequently use LLMs to bypass access controls and reach restricted resources, or to interact with internal services and manipulate sensitive data. Developers should be aware of the potential for SSRF vulnerabilities and take appropriate precautions.

LLM06:2023 - Overreliance on LLM-Generated Content

To prevent issues stemming from overreliance on LLM-generated content, organizations and users should take the following steps: independently verify content, consult alternative sources, ensure human oversight and review, clearly communicate the limitations of generated content, and use LLM-generated content as a supplement rather than a replacement. Examples of attack scenarios include news organizations publishing false information and companies relying on inaccurate financial data for critical decisions. Both situations can lead to the spread of misinformation and significant financial losses.

LLM07:2023 - Inadequate AI Alignment

Mitigating risks from inadequate AI alignment requires a multi-layered approach: clearly defining objectives and intended behavior, ensuring that reward functions and training data are aligned and do not encourage undesired or harmful behaviors, regularly testing and validating the LLM's behavior across various contexts and scenarios, and implementing monitoring and feedback mechanisms to continuously evaluate the LLM's performance and alignment. Additionally, proactively analyzing potential attack scenarios -- such as those involving inappropriate user engagement or system administration tasks -- can help reduce the risk of undesired or malicious outcomes.

LLM08:2023 - Insufficient Access Controls

To reduce the risk of vulnerability exploitation, developers must enforce strict authentication requirements and implement role-based access control (RBAC) to restrict user access. Access controls must also be established for LLM-generated content and actions to prevent unauthorized access or manipulation. Regular audits and updates are equally important to maintain security over time.

LLM09:2023 - Improper Error Handling

Improper error handling can allow attackers to discover sensitive information, system details, and potential attack vectors. To prevent this, proper error handling must be implemented to catch, log, and process errors gracefully. Error messages and debugging information must never reveal sensitive information or system details. Since attackers can specifically exploit LLM vulnerabilities through improper error handling, addressing this issue significantly reduces risk and improves system stability.

LLM10:2023 - Training Data Poisoning

Training data poisoning occurs when an attacker manipulates the training data or fine-tuning procedures of an LLM to introduce content that can compromise security, effectiveness, or ethical behavior. To prevent this, organizations should enforce data integrity, implement data sanitization and preprocessing, and regularly review the LLM. Monitoring and alerting mechanisms can also detect irregularities that may indicate malicious manipulation.

Conclusion

In conclusion, the OWASP Top 10 for Large Language Models provides an essential guide for understanding and preventing significant vulnerability types in AI applications built on large language models. Through proper sandboxing, input validation, authorization, and error handling -- combined with an understanding of the risks associated with training data poisoning and overreliance on LLM-generated content -- developers can keep their LLM implementations secure and ensure they function as intended.