Assuming Rhaegar didn’t marry Lyanna, would Jon then technically be a Waters or a Sand?

I’m not an expert in these matters, I would certainly defer to @nobodysuspectsthebutterfly on this, but my understanding is that the name is based on where someone is raised, not who their parents are. 

Hence why it’s Mya Stone rather than Mya Storm, even though Robert Baratheon is her father, or Aegor Rivers rather than Aegor Waters, even though Aegon IV was his father. 

So it would still be Jon Snow who knows nothing.