github.com
AI Goat
vulnerable LLM CTF challenges
Independent evaluation guide | 19 synced entries
How to evaluate LLM security and red-team tools
Test prompts, model behavior, guardrails, and application controls against abuse. Compare selection criteria here, then verify every claim against the linked first-party source.
Decision context
What does llm security tooling need to cover?
LLM security tools help teams find weaknesses that conventional application scanners cannot see. They exercise model behavior, system prompts, retrieval pipelines, tool calls, output handling, and policy controls under adversarial input. Common uses include prompt-injection testing, jailbreak evaluation, sensitive-data leakage checks, guardrail validation, and repeatable red-team campaigns. The tools listed here come directly from the community-maintained Awesome MLSecOps catalog rather than a paid placement program.
README-synced directory
LLM Security and Red Teaming catalog entries
Each listing preserves the neutral description maintained in the Awesome MLSecOps README and links to the resource's first-party source.
github.com
AI-Scan-Interceptor
Self-hostable DLP gateway for enterprise prompts to ChatGPT/Claude/Gemini (Squid + Go ICAP + mTLS, AGPL-3.0)
github.com
ARTkit
Automated prompt-based testing and evaluation of Gen AI applications
github.com
Damn Vulnerable LLM Project
A Large Language Model designed for getting hacked
gandalf.lakera.ai
Gandalf Lakera
Prompt Injection CTF playground
github.com
Garak
LLM vulnerability scanner
github.com
Giskard
Open-source testing tool for LLM applications
github.com
LangBiTe
A Bias Tester framework for LLMs
github.com
NeMo Guardrails
NeMo Guardrails allow developers building LLM-based applications to add programmable guardrails between the application code and the LLM
neuraltrust.ai
NeuralTrust
Tools to protect, secure and test GenAI Applications
list of various payloads for attacking LLMs collected in one place
github.com
Promptfoo Scanner
An open-source LLM red teaming tool
github.com
PromptInject
A framework that assembles adversarial prompts
github.com
PS-fuzz
tool for scanning LLM vulnerabilities
meta-llama.github.io
Purple Llama
Meta's umbrella suite of LLM safety and security tooling, including Llama Guard, Code Shield, and CyberSecEval
github.com
Pyrit
The Python Risk Identification Tool for generative AI
github.com
TrustGate
An open-source Generative Application Firewall (GAF)
github.com
Vigil
LLM prompt injection and security scanner
github.com
Whistleblower
Open-source offensive tool by Repello AI for testing LLM apps against system prompt leakage
No catalog entries match that search.
Selection framework
How should teams evaluate the options?
Evaluate an LLM security tool by matching its test library to your threat model and deployment architecture. Check whether it supports the models, APIs, RAG systems, and agent frameworks you use. Prefer reproducible test cases, machine-readable results, CI integration, clear success criteria, and controls for handling sensitive prompts. A broad attack library is useful, but evidence quality matters more than raw test counts. Review maintenance activity and licensing before putting any scanner in a production pipeline.
- 01Threat and model coverage
- 02Reproducible evaluations
- 03CI and reporting support
- 04Sensitive-data handling
Record evidence and limitations for each criterion. A catalog listing is a discovery aid, not a security certification.
Primary references