Best to get a direct drive turntable, as in a "DJ" turntable. If you are serious, Spend the $500 and get a Technics 1200. Youll have it for the rest of your life.
Check out
Turntable Lab in NYC for good prices.
When I buy music its mostly vinyl, but Im not a collector. My hobby is DJ'ing, so I mostly buy what I like or what I can throw on at a Party. That being said, I have no idea about indie rock/modern rock (Im a Soul/Funk/R&B/Rap/Jazz fan), so I cant recommend any stores per say. What I will say is buy what you like, NOT what is expensive and a collectors item. Much more enjoyable that way.
One spot Id recommend though, is this crazy homeless dude in the LES, on Chrystie St. (I Think), somewhere between Houston and Delancy. He has this tiny space and records all over the street. If you persevere and have an afternoon to look through the 1000's of LP's haphazardly strewn around in no particular order you can find some gems. I find great stuff on the street, and Ive also found awesome stuff in the garbage. Keep your eyes open. Searching out records is 85 percent of the fun.