I also suspect the size because a slug should not slide out with a dowel by hand. The bore must be oiled, not just the slug and you should have to pound it all the way. Brass rods the best, close to bore size.
I get a slug in the barrel and upset it with a brass rod at both ends. Yes, I can even do that with a revolver. You can wear the slug to a smaller size. Only pure lead should be used so it can upset to the groove bottoms. No spring back. I found round ML balls the right size work very well or pure sinkers, never an actual boolit.
let me find my ruler!