Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence. - Bo Peng; Daniel Goldstein; Quentin Anthony; Alon Albalak; Eric Alcaide; Stella Biderman; Eugene Cheah; Xingjian Du; Teddy Ferdinan; Haowen Hou; Przemyslaw Kazienko; Kranthi Kiran GV; et al. (CoRR, 2024)

GoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compression. - Daniel Goldstein; Fares Obeid; Eric Alcaide; Guangyu Song; Eugene Cheah (CoRR, 2024)

INTELLECT-1 Technical Report. - Sami Jaghouar; Jack Min Ong; Manveer Basra; Fares Obeid; Jannik Straube; Michael Keiblinger; Elie Bakouch; Lucas Atkins; Maziyar Panahi; Charles Goddard; Max Ryabinin; Johannes Hagemann (CoRR, 2024)