That’s pretty accurate. Prisons are very expensive – not only the structure, but also the food, water, bedding, clothing, waste disposal, and medical care for the inmates, the personnel needed to provide these things, keep order in the prison, and prevent escapes, etc.
Medieval governments usually didn’t have the fiscal or organizational capacity to run extensive prisons. So what you got instead were dungeons built into other facilities – the Tower of London was a royal residence, it was used as an armory and treasure-house because it was quite secure (hence why it still holds the crown jewels today), it was used for various government offices (the Royal Mint, and down in the basement they put the dungeons. And having visited quite a few castles when I was a youngun’, I can tell you, dungeons are small. Not only are the rooms quite cramped, but there just aren’t that many cells – maybe a dozen but usually less. Compare that to modern prisons with their hundreds if not thousands of inmates.
By contrast, exemplary physical punishment, whether we’re talking about crow cages or stocks/pillories or executions, are cheap. You only need one or maybe a few official torturers/executioners, some tools, a wooden platform, and a good spot at a public market so that people can watch. And once you’re done with the offender, you let them go or throw away the body.