Was there ever anything like the order of maesters in our own history? If not, how did nobles handle the education of their children?

Well, historically the role that the maesters play in Westeros was played by the Catholic Church in Medieval Europe. While priests were rarely as comprehensively educated as the maesters, they were the one (mostly) literate class in Europe, and the original universities of Europe were established to train them, and priests were often attached to noble households to provide advice, religious education, etc.