HU etiquette question

[ggbman]

So I am playing this guy heads up at 25/50 and im up about 10k on him. I'm sitting alone at another table, which he joins, and were end up being like 4 handed. I stack him with set over set, and a few minutes lter my friend calls and asks if i want to grab some dinner. we have been playing for about 45 minutes, so i sit out and tell the guy i'm going to dinner and may be back later.

He basically throws a hissy fit, accuses me of hitting and running, etc... Basically i don't feel that my job is to make him happy about the poker table, but i respect the game and don't want to be a dick. I told him he could have my aim name so we could schedule a rematch, but he was pretty cockish and went with the "i'm on here all the time, you know where to find me" line. So what do you guys think, is there anything wrong with what i did here?