Well, I certainly think there was an element of emotional infidelity, regardless of whether there was sexual infidelity. Clearly Naerys and Aemon were deeply in love, but it could have either been a case of courtly love or something more.
And for those who say that Aemon was too good a knight to break his vows, I would say only LANCELOT.