Preference Optimization This skill assumes already routed here because the data shape is preference pairs or unpaired thumbs-up/down feedback, not demonstrations (that's ) or a verifiable reward signal (that's ). What follows is method selection among the DPO family, the evidence for how much that selection actually matters, the production training pattern, and how to build the pairs in the first place. Input: a routing decision (preference optimization) plus preference pairs or unpaired feedback, usually from an SFT checkpoint. Output format: a validated method choice plus a config — the kwa…