Well, it’s both – he’s not actually Aegon the son of Rhaegar the crown prince, but he’s also Varys’ pawn.
I don’t think Varys would care about the lineage, per se. But I think he would care that his attempt to rescue a child from slaughter had been betrayed by the man he loved.