Supressed Circular Pattern lines showing in drawing. How can I stop that?

I have a part with two configurations. One of them has 4 Circular Patterns, the other one has only two.

But when I create a drawing and change between the configurations in the second one the annotation line of the first one are still showing and it makes the drawing illegible.

Is there a way around it?

SolidworksDrawings And Detailing