flopped a set on a dangerous board

I couldn't find the hand history for this hand but this is how it went approximately:

reads:

button - has been mixing it up pretty good. He is capable of raising on air I think. But most of the time a large bet means a big hand and his bluffs are small bets. I saw him check down top pair once. Once he made a min reraise after I min raised his pot bet on a paired board and he showed that he had nothing.

bb - New to the table, no reads yet.

blinds: .25/.5

button: $74
hero (sb): $53 hand:9 /images/graemlins/club.gif9 /images/graemlins/heart.gif
bb: $25

Preflop:
button calls $.5, hero raises to $1.5, bb calls $1.5, button calls $1

Flop: ($4.28) 9 /images/graemlins/spade.gifT /images/graemlins/spade.gifJ /images/graemlins/diamond.gif
button checks, hero bets $4, sb folds, button raises all in ~$72, hero... (owe all in ~$47)

What would you do?

I think you have to fold here...the few times you are ahead of someone playing AJ or a flush draw like an idiot don't make up for the times he has KQ and you're hoping to fill up. I think he might have a combo draw here, as well, which isn't something you particularly want him to flip up.