The normal difficulty with taking the Vale is the Mountains of the Moon acting as a barrier. If you can make a landing in the east, as the Andals did, you’ve got a lot of room to maneuver in the valley.
Now, the main difficulty is about resource management and timing: if you really wanted to win, you need to send at least 60,000 men and be ready to carry out some long sieges, which means you don’t have those men to deal with the Starks and the Ironborn and Jon Connington. So even in that scenario, it might be that the Lannisters and Tyrells win some initial victories, put the Eyrie and the other major castles under siege, and then run out of steam later on when they get distracted by other threats, giving the Valemen a chance to regroup.
On the other hand, I don’t know if anyone other than Lysa Arryn would fight to save Littlefinger. If the King on the Iron Throne sends a public letter condemning Littlefinger as a kingslayer and traitor, you’re probably going to get the Lords Declarant Plus declaring that Lady Arryn is being held hostage by the evil Littlefinger, let’s go save her.