I think you played it fine. If he was a donkey who made a habit of pushing, then it's an easy call. But not here.

With a higher percentage of my stack in the pot, I would've check-raised all in. But not when you can get away from the hand and still be viable.

I'm guessing he has an overpair and put you on either top pair or a flush draw. Or he has two overcard Hearts.