I’m not so sure it would be. Look back through the history feudal politics – even people with virtually unanimous reputation found enemies of their enemies, simply because their removal would upset the status quo, or empower someone that someone else didn’t like, or people felt it would set a bad precedent.
After all, if you can wipe out the Freys for being bad vassals, what does that mean for the Brackens or the Blackwoods? And I’m sure the neighbors of their neighbors don’t like the idea of rivals getting more powerful from dividing up the Frey lands.