Description
Spikee, developed by Reversec, is a Simple Prompt Injection Kit for Evaluation and Exploitation. It is designed to assess the susceptibility of large language models (LLMs) and their applications to targeted prompt injection attacks. Unlike existing tools that focus on generic jailbreak scenarios, Spikee prioritizes cybersecurity threats such as data exfiltration, cross-site scripting (XSS), and resource exhaustion. These scenarios are based on real-world outcomes and pentesting practices.
Spikee provides a practical tool for testers to generate customizable datasets, apply static evasion plugins or dynamic attack strategies, and test targets including LLMs, guardrails, and entire applications. It integrates with tools like Burp Suite, allowing users to analyze results and false positive rates for guardrails.
The tool can be applied across the LLM application security pipeline to evaluate and enhance resilience against prompt injection attacks. Key use cases include testing LLMs in isolation, using custom datasets, evaluating individual guardrails, and assessing the entire LLM-driven application pipeline.
Spikee's v0.2 adds support for dynamic attack strategies and a flexible judge system for evaluating attack success. It also offers detailed setup and usage instructions, including dataset generation, target testing, and result analysis. Users can generate datasets from seeds, customize them with plugins and filters, and run tests against targets like GPT-4o using dynamic attacks.
Future developments for Spikee include integrating with pentester tools, enabling vision attacks, improving the judge system, and expanding libraries with new jailbreaks and attack techniques. Contributions from the community are welcome to help evolve the tool based on emerging research and feedback.
Spikee's Core Features
Customizable dataset generation
Static evasion plugins
Dynamic attack strategies
Integration with Burp Suite
Guardrail false positive analysis
Flexible judge system
Cybersecurity-focused scenarios
Real-world outcome-based testing
LLM application security pipeline evaluation
Support for dynamic attack strategies
How to use Spikee?
Initialize: Install Spikee via PyPI and set up workspace
Generate Dataset: Customize with seeds, plugins, and filters
Test Target: Run tests against LLMs or guardrails
Analyze Results: Calculate metrics and generate reports
Spikee's Use Cases
- LLM Isolation Testing
- Custom Dataset Testing
- Guardrail Evaluation
- Pipeline Assessment
- Cybersecurity Threat Analysis
