How did the Roman Emperors choose their heirs? How efficient was this system of choosing successors?

It varied enormously. There was quite a bit of inheritance by blood relation, there was quite a bit of inheritance by adoption, and quite often you had acclamation/election by either or both the Senate or the Army as well. 

Sometimes it worked very smoothly, and sometimes you got elite coups, and sometimes you had terrible civil wars.