It’s hard to say, because we don’t know what the parameters for the Wall’s magical defenses are, and we do know from Melisandre and Storm’s End that there are way to bypass them.
- It could be that the Horn was their trump card.
- It could also be the case that the wight attack in AGOT and/or the attack on the Fist in ASOS was an attempt to “dead-drop” assets behind the Wall in order to de-activate them or somehow take advantage of the one-way block on magic.
- It could be the case that the Wall’s defenses require a Night’s Watch to be present for them to work – and killing off the leadership or reducing the overall numbers of the Watch or getting the wildlings to do that for them would cause the magical defenses to drop.