Motion towards the pot. Question.

Heads up, NLHE. You flop top set. You bet, opponent calls. You turn the boat and have the imortal nuts. You bet, opponent calls. River, brick. You make a value bet. Your opponent thinks for a minute. He happens to have the exact amount for a call covering his hole cards as a card protector. He moves the chips off of his cards, slightly towards the pot, holds his cards facing him in the air, and looks at you. You flip over your cards assuming that he just called and wants you to show first.
He says he never verbally declared he was calling and says your hand is now dead.
When you tell him he put the chips forward for the call he says he was just taking them off his cards. Even though he moved them forward, towards the pot and not sideways.
I'm asking this here because it happened in a home game and there was no floor manager to rule on it.
What is the normal ruling here?