I think there are two main, complementary explanations to how Renly figured it out:
- Renly was a political actor in the royal court. One of the main activities of someone in his position is to closely observe and/or spy on the other main political actors. Given that Jaime and Cersei are so indiscrete that Varys, Pycelle, Littlefinger, Stannis, and Jon Arryn all found out, it was eminently possible for Renly to have found out by either observing/spying on Jaime and Cersei directly, or by observing/spying on Varys, Pycelle, Littlefinger, Stannis, and/or Jon Arryn and finding out what they found out.
- Renly knows what Robert’s kids look like. Edric Storm is Renly’s ward and living in his own castle, and he looks like the spitting image of Robert Baratheon. Moreover, if Renly is spying on the other political actors, he might well find out about Stannis and Jon Arryn investigating Robert’s other bastards in King Landing, or Jon Arryn inquiring about Pycelle’s book, or Littlefinger whispering about Robert’s kids not looking like Robert, etc. Once you’re there, it’s not hard to put two-and-two together.