Interesting A-5 hand

[Yads]

So I was playing a bit of 2/4 A-5 after a bit of a haitus from the game. Man I forgot how much I enjoy playing low ball. Also this is the first time playing 1-5 in a while as I normally play 2-7. Anyway the following hand came up and I thought it was interesting. Let me know what you think.

It's 5 handed and I'm on the button with 97654. Folded to me I put in a raise. SB folds, BB calls.

1st Draw
Pot is 4.5 SB
Villian draws 4, I decide to stand pat. I'm guessing this is a no brainer, but let me know if you think otherwise.
Villian checks, I bet, he calls.

2nd draw
Pot is 3.25 BB
Villian draws 2, I stand pat. Should i have broke this hand and drawn to a rough 7? Villian checks, I bet he calls.

3d draw
Pot is 5.25 BB
Villian draws 1, I stand pat. Villian has 20 outs against me if he is drawing to an 86. I'm a slight favorite, or am I thinking about this wrong? Villian bets and I call. Obviously this is a terrible call on the end as villian is not bluffing at a pat hand often enough to make the call profitable.

So aside from the atrocious river call anyone play this differently?