There are a lot of old people in Las Vegas. The average age of a LV visitor in 2004 waas 49 (down from 50 in 2003). If you're from Boston, with an average age of 36 (yeah, college town) that's something of a change.
But chances are you won't be doing the same thing as the retirees, and not in the same places. The Hard Rock pool will look a lot different from the Stardust pool. Those are extremes, but do your research and you'll be fine.
Another Vegas statistic - 93% of 2004 visitors polled were "very satisfied" with their trip. Of the remaining 7%, only about 3% said the reason they were unhappy was "nothing but gambling." So plenty of people are finding plenty to do.
source - LVCVA
You'll have plenty of things to do other than gamble.