All repositories

Securityethz-spylab

agentdojo

A dynamic environment for evaluating attacks and defenses against LLM agents. It runs an agent on user tasks and checks whether a malicious instruction gets through.

Who it is for

For researchers and engineers comparing agent defenses against prompt injection.

How to start

  1. Install: pip install agentdojo
  2. Run a single test: python -m agentdojo.scripts.benchmark -s workspace -ut user_task_0
  3. For the full benchmark, pass a model with --model as shown in the README

Steps are taken from the README. Check the current version in the repository before running them.

Stars over the last 30 days

+105Oct 22 — Nov 19
9491,054

Author's description

A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents.

data-privacy-stack

presidio

A framework that finds and masks personal data in text, images and tables. It detects the data, then redacts or replaces it so it can be safely sent to a model.

12Kstars+423 in 28 dPython

NVIDIA

garak

A vulnerability scanner for LLMs: it runs a model through a set of probes for prompt injection, data leakage, toxicity and other failures. Works as a command-line tool.

10Kstars+365 in 27 dPython

guardrails-ai

guardrails

A Python framework that checks LLM inputs and outputs with ready validators from Guardrails Hub and helps produce structured answers. It catches risks before a reply reaches the user.

7.6Kstars+61 in 28 dPython

NVIDIA-NeMo

Guardrails

NVIDIA's toolkit for programmable rails in LLM-based conversational systems. Rules live in config and control what the bot talks about and does.

7.5Kstars+154 in 28 dPython

microsoft

PyRIT

Microsoft's framework for proactively finding risks in generative AI systems. It helps automate red teaming against models and applications.

4.9Kstars+213 in 27 dPython

meta-llama

PurpleLlama

Meta's generative AI safety project: the Llama Guard and Prompt Guard filter models, the Code Shield scanner and the CyberSec Eval test suites. It pairs defense with attack-based testing.

4.5Kstars+39 in 28 dPython