If he does this to you literally every time from the SB, you should consider 3-betting PF. Your queen may have high card showdown value. I don't mind calling with this sort of hand as long as you've played enough hands with villain to know he doesn't simply do this with the top 20 % of his hands.

On the flop, your high card may be good and you have a gutshot. This clearly calls for a raise. If you can't raise the flop, you need to fold. If he 3-bets you then it's probably time to fold if he bets out again on the turn. As played, you need to check behind on the river once your 4s hit. You now beat his high card hands but lose to just about everything else; there's no value here and you're never folding out anything you beat.