Safety

BOLD

BOLD measures biases in open-ended language generation. This import preserves HELM's published fraction of toxic completions for each request, with lower values indicating less toxicity.

2,028items

42subjects

unknownlicense

safetydomain

textmodality

item-level responses released

Saturation status: No

Full data on Hugging Face Back to the gallery

Paper and credit

2021

BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation (opens in a new tab)

Original source

Authors

Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna +3 moreShow all

01Jwala Dhamala
02Tony Sun
03Varun Kumar
04Satyapriya Krishna
05Yada Pruksachatkun
06Kai-Wei Chang
07Rahul Gupta

Citation

BibTeX citationShow

BibTeX

1@inproceedings{dhamala2021bold,
2  title={BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation},
3  author={Dhamala, Jwala and Sun, Tony and Kumar, Varun and Krishna, Satyapriya and Pruksachatkun, Yada and Chang, Kai-Wei and Gupta, Rahul},
4  booktitle={Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency},
5  year={2021},
6  publisher={ACM},
7  pages={862--872},
8  doi={10.1145/3442188.3445924},
9  url={https://doi.org/10.1145/3442188.3445924},
10}

Response matrix

the original evaluation of Dhamala et al. covered 5 model configurations over all 23,679 prompts

Loading response matrix…

toxic continuation (1)no toxic continuation (0)Unobserved

Scale: 1 = toxic continuation · 0 = no toxic continuation