You are getting 7:2 on the calldown.
Roughly 3.5:1 and you are drawing dead to a set/made flush and will lose 20% of the time that you are ahead to a one card flush.

I fold here, but maybe I underestimate how often he holds a pair + Xd.

True