The probabilty of 'other' occuring is not 0.96 every time. You are assuming the events are independent. But the probabilty of the events depend upon what happened before, so you need to account for this. In your coin flip example the events are independent, the coin has no 'memory' so you were correct in using 0.5 each time.

For exmaple if 1=3rd then P(2=other) = 48/49 = 0.98 (not 48/50)

The 3 answers I get are all 0.038367
So 3*0.038 = 11.5%

This is still wrong because the 'other' cards can still produce a full-house. So 11.5% - P(full-house) = 11.5 - 0.74 = 10.76% to hit only a set.
Please correct me if you think this method is wrong.
Thanks,
Mike