Ok, is it possible that your boolits might be a little oversized, or the neck portion of your chamber might be a little tight? If not, that slight flaring of the case mouth might simply just be too much for the drop-fit. Factory loads don't have that feature, therefore they won't have that problem.
Since you're getting best accuracy that way and the rounds chamber with only slight resistance, I'd be tempted to live with it as long as there are no other problems. Fire off a few magazines full, slow-fire and rapid-fire, see if they feed reliably. If they do (again, if it was my rifle) I might leave well enough alone and enjoy those accurate handloads.