I think he drew more than a little bit from his research on the antebellum South for Fevre Dream, but there are other influences in no small part because there were other historical parallels available: Volantis has some strong influences from Constantinople, Braavos borrows from both Venice and Amsterdam, etc.
Slavers’ Bay is interesting, because it borrows heavily from Rome – the lockstep legions, the toga-like tokars, gladiatorial arenas, etc. – but also from Carthage, given its extended conflict with the Valyrian Freehold, the Harpy standing in for Baal Hammon, etc.