SKU PRF-3004 · Sold by External
PKU-SafeRLHF
Product specifications
| SKU | PRF-3004 |
|---|---|
| Data type | Preference data |
| Volume | 83.4K preference entries (Q-A pairs, each with two ranked responses) |
| Size on disk | Not published by source |
| Format | JSONL (per-model train.jsonl / test.jsonl; auto-converted to Hugging Face Parquet) |
| Access model | PUBLIC LICENSE |
| Pricing | Free · open dataset (HF) |
| Quality score | — |
| License | CC-BY-NC-4.0 |
PKU-SafeRLHF is a preference dataset built by the PKU-Alignment team for multi-level safety alignment of LLMs. Each entry pairs a prompt with two responses (generated by Alpaca-7B, Alpaca2-7B, and Alpaca3-8B) and provides two-dimensional human annotations: a helpfulness ranking (better_response_id) and a harmlessness ranking (safer_response_id). Every response also carries a binary safety flag, a severity level (0-3), and boolean labels across 19 harm categories (e.g., Economic Crime, Mental Manipulation, Violence). Annotators enforce a logical rule that harmless responses are always ranked above harmful ones on the harmlessness dimension. The dataset card reports 83.4K preference entries and is released under CC-BY-NC-4.0; the auto-generated dataset viewer shows 82.1K rows in the default config (73.9K train / 8.21K test). It is a sibling of the BeaverTails and PKU-SafeRLHF-QA datasets and is intended for research that makes models less harmful.