Man-at-arms historically was a term to describe a professional fully-armored heavy cavalryman, someone who had been trained in the arms and armor of a knight, even if they weren’t a knight themselves.
So hardened leather was a step below them – it would have been chainmail in the early Middle Ages, changing to a mix of chain and plate by the late 14th century, and then full plate from the 15th century on.