did I just get scared off the best hand?

I play it the same, unless I know the guy with the small hearts is absolutely useless. I think there's a danger here that we find hands we can beat, or play against, and put both villains on them, to justify playing on. The three of hearts is probably the worst card in the deck for the other villain to get from your point of view here, and I would guess that more often than not, you will get raised on fifth.

I think folding, if seat one has two pair and the other guy is a fool, is not that bad an error. On the other hand, playing against any of trip nines, a flush or a straight is going to be costly, especially if you improve a bit.