TRL Each method pairs a class with a dataclass. Configs extend , so all of its arguments work in any trainer config. | Trainer | Dataset type | |---|---| | | language modeling or prompt-completion | | | preference (chosen/rejected pairs) | | | prompt-only + reward function(s) | | | prompt-only + a teacher model (on-policy distillation) | | | unpaired preference (per-sample bool label) | | | preference (chosen/rejected pairs); trains a scalar reward model, not a policy | Many more trainers (OnlineDPO, ORPO, CPO, GKD, …) live in with unstable APIs: https://huggingface.co/docs/trl/experimental o…