Yes, I think you've got a valid complaint. The other bettor has heard this dealer threaten to cost you money, so collusion is implied, although the cardroom should find the non-dealer blameless.

What you're basically describing is a "monkey in the middle" strategy, with you as the chimp. Rather than report it, I'd use it to your advantage. Wait for the strong hand and trap the other player.

Having said all of this, I find this scenario at best to be extremely rare. Even a drunk clown isn't going to piss money away just to get back at you. You two must have some real issues.