(5) AQs on the button

PokerStars Game #3142525198: Tournament #15543499, Hold'em No Limit - Level I (10/20) - 2005/11/24 - 01:27:20 (ET)
Table '15543499 1' Seat #5 is the button
Seat 1: badbeatbean (1500 in chips)
Seat 2: roadking4502 (3030 in chips)
Seat 4: SaldAjeweLa (2280 in chips)
Seat 5: ShrimpHouse (1460 in chips)
Seat 6: whamo (1480 in chips)
Seat 7: jayflee (1500 in chips)
Seat 8: ijmorris (750 in chips)
Seat 9: Anduriel (1500 in chips)
whamo: posts small blind 10
jayflee: posts big blind 20
*** HOLE CARDS ***
Dealt to ShrimpHouse [Ad Qd]
ijmorris: folds
Anduriel: folds
badbeatbean: folds
roadking4502: raises 40 to 60
SaldAjeweLa: raises 40 to 100
ShrimpHouse: calls 100
whamo: folds
jayflee: folds
roadking4502: raises 80 to 180
SaldAjeweLa: calls 80
ShrimpHouse: calls 80
*** FLOP *** [5c Kd Jh]
roadking4502: checks
SaldAjeweLa: checks
ShrimpHouse: checks
*** TURN *** [5c Kd Jh] [Qh]
roadking4502: bets 140
SaldAjeweLa: calls 140
ShrimpHouse: ?

I kind of got trapped into playing this hand as I felt I needed to at least see the flop with that much invested. The flop comes and gives me a backdoor flush and a one sided straight draw which isnt that spectacular.
Once the queen rips on the turn I am still worried about a set. What would you do? Was it too early to play this hand?