This is what GRRM has said on the matter:

“Well, the short answer is that the laws of inheritance in the Seven Kingdoms are modelled on those in real medieval history… which is to say, they were vague, uncodified, subject to varying interpertations, and often contradictory.

A man’s eldest son was his heir. After that the next eldest son. Then the next, etc. Daughters were not considered while there was a living son, except in Dorne, where females had equal right of inheritance according to age.

After the sons, most would say that the eldest daughter is next in line. But there might be an argument from the dead man’s brothers, say. Does a male sibling or a female child take precedence? Each side has a “claim.”

What if there are no childen, only grandchildren and great grandchildren. Is precedence or proximity the more important principle? Do bastards have any rights? What about bastards who have been legitimized, do they go in at the end after the trueborn kids, or according to birth order? What about widows? And what about the will of the deceased? Can a lord disinherit one son, and name a younger son as heir? Or even a bastard?There are no clear cut answers, either in Westeros or in real medieval history. Things were often decided on a case by case basis. A case might set a precedent for later cases… but as often as not, the precedents conflicted as much as the claims.”

Keep in mind, Randyll Tarly wasn’t proposing to write a will that bypassed Sam in favor of Dickon, or any such legal manuever. He was straight-forwardly threatening to murder Sam if he didn’t comply. 

So yes, potentially Sam could have gone to Highgarden and protested that his father was not only breaking the laws of inheritance but was an attempted kinslayer and murderer on top of that. I doubt that Mace Tyrell would have been particularly interested in championing Sam’s claims, however.