Are we including the Ur-Text? Because then I think AGOT counts as the biggest retcon.
My biggest retcon peeves are the small things: Bloodraven being Maekar’s Hand, Aenys I not being married to his sister (although I may be mis-remembering things), Aegon V not fighting the War of Ninepenny Kings, etc.