I know the rule, but was I right?

[Loci]

It's a rule for a reason. The dealer shouldn't have let it happen in the first place. They're underpaid and under appreciated, they do make mistakes and rules like that protect you for it. Don't feel bad about hurting the guys feelings and taking his money. If it's really weighing on you, give him a bet back.