I’m probably not the best person to ask about this (you might want to ask @goodqueenaly or @nobodysuspectsthebutterfly), but as far as I know, we don’t know exactly how such a connection came to pass.
All we know of the Tarth’s family tree is that “House Tarth of Evenfall Hall—an old family of Andal descent that boasts of ties to the Durrandons, the Baratheons, and more recently to House Targaryen.” One theory I’ve heard is that the Targaryen connection was one of Egg’s sisters who had an amour with Dunk and produced a child that was born legitimate, what we could call the Ossifer Plumm Theory, but it could have happened some other way.