Harry was absolutely an ass to Alayne, the only difference between him and other Vale lords is that they might be refined enough in their ettiquette as to be an ass behind her back.
As to the other Vale lords, only the Waynwoods get to make the call, so they might be pissed but they can’t do much about it.