Probably a mix of genes and something else? We know from Rhaenyra and Dany that Targaryens can produce deformed and stillborn offspring, hence why the Targaryens practiced eugenics as well as incest.
At the same time, as with Dany, outside factors (magic, poison, etc.) can cause that tendency to be expressed where otherwise the fetus would have been healthy.
Here’s a third possibility: maybe Maegor died after the Trial By Seven, and Visenya brought him back to life as a revenant?