Yes, geographically, it’s a bit odd.
But Lannisport seems to be a case of urban development around industry – namely that it’s the major harbor near a really really big mining, smelting, and smithing complex that includes gold, but also steel and likely copper and silver as well. Add to the industrial population the people who work in services around that industry (you’ll need assayers for the gold, you’ll need lots of guards and locksmiths for security, you’ll need merchants to handle transactions and clarks to write things down, you’ll need warehouse workers to store the stuff, you’ll need bakers and butchers and grocers to feed the population, you’ll need tavernkeepers and bartenders to get them drunk, you’ll need a city watch to keep order).
Then because it’s the major port near the city, and sea transport offers advantages in terms of transporting your heavier cargo, you have longshoring and shipbuilding and local merchants who decide to branch out from just handling local commerce, etc.
Then you add on that, given that gold and gold products are a high-value commodity, and that a lot of places need it for their currency as well as for decorative purposes, merchants from Oldtown and points beyond are going to want to deal directly with the producer rather than deal with the markup from middlemen. And so the city keeps growing, despite being on the wrong side of the continent.
Is very true. In the real world the city of Potosí, which was for all intents and purposes in the ass end of the world, had at one point around 200,000 inhabitants thanks to the silver mine.
Casterly Rock would be like that on steroids, plus providing the processing and shipping for all the gold and silver produced in the rest of the Westerlands.
That’s an excellent comparison!