Well, primarily what the lord did was to take all the rent and taxes, the free labor, the extra income from mills and bridges and the like, and all the other feudal privileges to which they felt entitled, and not work for a living and instead focus on hunting and jousting and war and politics and really stupendous levels of conspicious consumption. 

In terms of how easy it would be to get rid of them, well, that was most of what the 14th century peasant revolts were all about. When you look at how the revolts start, one of the first targets of the peasantry were court records – chiefly manorial court records that were the only source of information about who was a serf and who wasn’t, who owed feudal obligations and who didn’t. Even before they got to the point of capturing capitol cities and demanding royal charters of liberty, the peasants would just burn the records that said they were serfs or villeins or owed any kind of free labor and then dare their landlords to prove that they weren’t free men. 

So ultimately, I’d say the main difficulty with getting rid of the nobility is that they were perfectly willing to 1. renege on any agreement or oath made to rebellious peasants, 2. ignore a flag of truce and murder the peasants’ leadership, 3. call out the army and kill thousands of people to restore compliance with the ancien regime.