A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity. - Andrew Lee; Xiaoyan Bai; Itamar Pres; Martin Wattenberg; Jonathan K. Kummerfeld; Rada Mihalcea (ICML, 2024)