Reward Modeling

RewardBench

Original per-item reward-model preference evaluations on the RewardBench core task bank.

4,045items

167subjects

Task database: ODC-By-1.0 (captured LICENSE.md); evaluator code: Apache-2.0. The result card releases native scores for further research without a separate license identifier.license

reward_modelingdomain

textmodality

item-level responses released

Saturation status: Yes

Full data on Hugging Face Back to the gallery

Paper and credit

2024

RewardBench: Evaluating Reward Models for Language Modeling (opens in a new tab)

Original source

Authors

Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda +8 moreShow all

01Nathan Lambert
02Valentina Pyatkin
03Jacob Morrison
04LJ Miranda
05Bill Yuchen Lin
06Khyathi Chandu
07Nouha Dziri
08Sachin Kumar
09Tom Zick
10Yejin Choi
11Noah A. Smith
12Hannaneh Hajishirzi

Citation

BibTeX citationShow

BibTeX

1@misc{lambert2024rewardbenchevaluatingrewardmodels,
2      title={RewardBench: Evaluating Reward Models for Language Modeling},
3      author={Nathan Lambert and Valentina Pyatkin and Jacob Morrison and LJ Miranda and Bill Yuchen Lin and Khyathi Chandu and Nouha Dziri and Sachin Kumar and Tom Zick and Yejin Choi and Noah A. Smith and Hannaneh Hajishirzi},
4      year={2024},
5      eprint={2403.13787},
6      archivePrefix={arXiv},
7      primaryClass={cs.LG},
8      url={https://arxiv.org/abs/2403.13787},
9}

Response matrix

Loading response matrix…

lowhighUnobserved

Scale: Grades: 0, 0.5, 1