I keep getting hole wizard picking up on points that I make for reference. It's becoming a hindrance.

In Inventor you could just swap between the two option i.e. 'right-click' on the point and choose what you want it to do. Is this option available on Solidworks? If not, how can I stop hole wizard from grabbing my point?

(and yes, I do want to use points)

Thanks