Jlyon,
I shot about 1600 rounds of cast boolits through my BHP. I did not cast them, they were given to me by a friend, who also let me use his Lubersizer and showed me how to do them. (Early 1990's)
I found by adjusting my load, and setting my dies to mimic factory (jacketed) bullets, they shot fine. They were RN, Lyman #2, and I believe they were 125 or so grains. (It was almost 20 years ago now.)
They did tend to dirty up the action something awful (probably the lube used). But they fed and shot as well as my poor aim with a pistol would allow.
But I tend to focus on one firearm at a time. Or one pistol, and one rifle. Until I develop a pet load for that particular arrangement of components.
So I know it can be done. Slow down, and you will probably speed up.