Yes it is. This is not a world where there are clothing stores on every high street – clothes are made by hand, predominantly within the family. Moreover, this is a world in which people live in very small, close-knit communities and are highly suspicious of strangers.
Especially in the North, where people are highly cognizant of the Watch and deserters, no one is going to buy the old “oh no, robbers took my clothes” line coming from a stranger.