Safety
XSTest
XSTest checks whether models refuse harmless prompts that merely sound dangerous, pairing them with genuinely unsafe lookalikes so that both excessive caution and unsafe compliance show up. Recorded scores retain intermediate judge ratings; 1 indicates the best rubric score and 0 the worst.
450items
81subjects
unknownlicense
safetydomain
textmodality
item-level responses released
Saturation status: Yes
Response matrix
Responses come from Stanford CRFM's HELM Safety leaderboard (v1.17.0), run without a system prompt; the original evaluation of Rottger et al. labels the same prompts on a 4-level ordinal scale from two human annotators and a GPT-4 judge. Complements the standalone xstest build.
Loading response matrix…
lowhighUnobserved
Scale: Score in [0, 1]