(About Freys) How come? Walder always talked of sending his family away because there are too many.

He does, but in the context of marriage alliances which strengthen the family. But if we look at the Red Wedding, he couldn’t have done it without dozens of family members he could trust implicitly to help carry it out, or to marry and/or occupy Riverlands castles in the aftermath to secure his position.