The Westerosi seem to be both religiously conservative while also surprisingly tolerant of prostitution (brothels operate openly, pay taxes and its not even a secret that a member of the small council operates one.) Is this realistic? how freely could sex workers work in medieval/renaissance europe?

Yep, it’s pretty realistic. There’s a pretty wide literature on the history of medieval prostitution, which is very interesting if you want to know more about the history of gender, sexuality, and culture, and I haven’t read more than just the basics. 

Here’s the short and simplified version: in Medieval Europe, prostitution was seen as a necessary evil, something that would prevent adultery, rape, masturbation, and sodomy (which were seen as more important sins). The Church wasn’t super happy about it, and preachers would try to persuade prostitutes to reform, but the Church wasn’t about to ban it (especially since a lot of clergymen made use of prostitutes since they weren’t allowed to get married). 

What did happen is that prostitution was regulated: in some places, it could only be done outside the city walls; in other places, it was restricted to certain streets or neighborhoods; in some places, there were civic brothels that were given a monopoly on the trade.