They did try:

Farther south, the wealthy harbor town of Gulltown on the Bay of Crabs was ruled by Osgood Shett, Third of His Name, a grizzled old warrior who claimed the ancient, vainglorious title King of the True Men, a style that supposedly went back ten thousand years to the Dawn Age. Though Gulltown itself was seemingly secure behind its thick stone walls, King Osgood and his forebears had long been waging an intermittent war against the Bronze Kings of Runestone, a more powerful neighbor from a house as old and storied as their own. Yorwyck Royce, Sixth of That Name, had claimed the Runic Crown when his sire died in battle three years previous, and had proved to be a most redoubtable foe, defeating the Shetts in several battles and driving them back inside their town walls.

Unwisely, King Osgood turned to Andalos for help in recovering all he had lost. Thinking to avoid the fate of Shell and Brightstone, he sought to bind his allies to him with blood in place of gold; he gave his daughter in marriage to the Andal knight Gerold Grafton, took Ser Gerold’s eldest daughter for his own bride, and married a younger daughter to his son and heir. All the marriages were performed by septons, according to the rites of the Seven From Across the Sea. Shett even went so far as to convert to the Faith himself, swearing to build a great sept in Gulltown should the Seven grant him victory. Then he sallied forth with his Andal allies to meet the Bronze King.

King Osgood won his victory, as it happened, but he himself did not survive the battle, and afterward it was whispered amongst the Gulltowners and other First Men that it was Ser Gerold himself who struck him down. Upon his return to the town, the Andal warlord claimed his good-father’s crown for his own, dispossessing the younger Shett and confining him to his bedchamber until such time as he had gotten Ser Gerold’s daughter with child (after which the father vanishes from the pages of history).

When Gulltown rose against him, King Gerold put down the protests brutally, and soon the gutters of the town ran red with the blood of the First Men … and women and children as well. The dead were thrown in the bay to feed the crabs. In the years that followed, the rule of House Grafton remained uncontested, for (surprisingly) Ser Gerold proved a sage and clever ruler, and the town prospered greatly under him and his successors, growing to be the first and only city of the Vale.

If the Andals had waited a few more years, the Royces might well have captured Gulltown.