To flip or not

This is quite early in the 20/2 180 player tourney. I haven't played a single hand yet. A short-stack (200T) goes all-in and I have 33. I called, and lost when he showed an ace crap and caught an ace. After a few hands, he pushes again this time ofcourse with 400T. I have 99, call and lose again when he shows AJo and catches a jack.

Am I wrong in calling these all-ins? The guy was playing like crap, and thought I would take a chance. Should I pass up these kind of situations early? I was not out, but I don't like being short-stacked early in the tourney.

Comments?