I get the political logic, but that’s not what happened. Tywin had lethal custody of Elia’s children before either Robert or Jon Arryn arrived in King’s Landing, and the entirety of the Royal Navy was standing in between them and Viserys on Dragonstone, so it’s not that easy.