Looking at all the maps of Westeros we’ve seen, there are a lot less castles than one might expect (considering the size of the continent). Do you think we only see the bigger castles and, if so, what is the cut-off point at which the castles are considered too small to not be included? Would these smaller keeps all be run by masterly houses like the Glovers or how would they fit into the feudal system present in the books?

I wouldn’t necessarily say we only see the bigger castles – I think we’re mostly given the castles that have been mentioned in the text, and even then they generally don’t map all but the most frequently mentioned. 

An interesting point of reference here would be to compare the Riverlands in the official Lands of Ice and Fire to say, Ser Mountain Goat’s “speculative map”:

The sheer density of the latter compared to the former speaks for itself.