Well, sieges and seasonal warfare had strong interactions: the foks inside would hope that supply problems would make the larger army go away before their stores inside ran out, the folks outside would try to feed themselves as best they could on the surrounding areas. But it’s also true that because of the size disparity between a castle garrison and an army, you could downsize your army to just enough men to keep them bottled in if the levies were needed in the field (this is another reason, btw why kings started to rely more on professional soldiers).