Would this Simulation result cause concern?

I am not a trained Engineer. I am learning to use simulation to do some design work before a final review. I understand that simulation will give some false results around edges. Below I have two screen shots of a simulation result. It is of a steel pin being side loaded at the tip by 400 lbs with fixtures at the top around two cylindrical faces. The simulation results completely makes sense with reality. I have ISO clipping turned on on the safety factor screenshots to display less than 1. You can see that safety factor is well below 1 in some very small areas on that fillet. Is this a concern? Does this look like a "false" result? Most of the pin in that area is well above 1.

Basically, how would I tell if the result are predicting true failure? My guess would be that when the majority of the pin at the stressed point in the neck is below 1 it would fail.

The Simulation set up:

Displacement result:

Safety factor:

Safety factor zoomed in:

SolidworksSimulation