WARDBREAKER: testing whether LLM guardrail bypasses actually work

A local fuzzer for open-weight guardrail classifiers (Llama Guard 3, Prompt Guard 2). It rewrites and re-encodes prompts in many ways and measures how often the guard gets fooled, per attack family, plus how often it over-blocks harmless text.

Most guardrail tests stop when the classifier says “safe.” WARDBREAKER goes one step further and sends the bypass to the model behind the guard to see whether the attack still does anything.

Finding: 77% of the prompts that flipped the guard didn’t survive the model behind it. So raw “bypass rate” numbers overstate the real risk by a lot, and you need the second check to know which bypasses actually matter.

Zero dependencies, runs fully local.
Code: https://www.twine.net/signin