Smart Component Help

I am trying to create a smart component and I cannot get it to work 100%. My issue is that when I place the smart component on the opposite side of the panel, it will not work. When the component is placed on the same side of the panel it works fine. Please see attached. I am going to guess that having the cut extrude in the assembly is the incorrect way to go about this and would need to do it at part level.

SolidworksAssemblies