Non-formal education

[Zygote]

I'm looking for advice in this thread about my education.

Little backround:
I did two years of university towards a commerce degree, but dropped out to play poker. I also hated what I was studying and after giving a bit of philosphy a try, i got the final push and left. Poker is going well and i am only looking to learn as a hobby, not as preparation for another career. I hate formal learning and need to work on my on terms, plus i need time for poker. So, going back to school appears like a poor option for me.

Recently I have come to realize that I am very interested in physics and would like to be able to understand the math at high levels. So far i've bought myself a selft-teaching guide that goes over the fundamnetals. I have a brief history of time and just got the universe in a nutshell, but they lack the depth that i am seeking. I also have a linear alegebra book and calculus book from university and i'll obviously purchase whatever is necessary. Heres my question, though: is it possible for me to independtly learn physics at high levels? I don't see why i can't, but maybe i'm overly optimistic.