Hi,

I got point stress of some >600N/mm2 on shaft edge where notch is while in the middle there is basically almost no stress...

How should I deal with this situation?

It's the similar case as in this situation: How are notch or stress concentration factors in FEA of shaft strength considered? 

Considering the shaft diameter I would never expect material failure. So should this be ignored?