By putting in about a half-pot sized bet on the turn you could have saved yourself alot of grief. He may have folded, and if he raised, you could have folded.
As for what to do now? I would fold, he may not have you beat, but I wouldn't risk it here. His large river bet may not be an attempt to scare you off, it could be a value bet. I've seen many players at these low levels call with weak hands. Just remember how the hand was played, and use it to set him up later.

Enjoy Life!