I use JB Weld inside the slipping handles and have pretty much decided I'm going to go through an apply to all the handles. The pain in the "other end" of slipping handles would count as minor compared to having a mold drop and hit the concrete floor. Had one slip off when I wasn't watching it closely enough (had a good groove going) fortunately I caught it as it slipped of the end. Also fortunately I was wearing gloves so I didn't brand myself with NOE on my hand.
I debated roll pin. Would take some strain and provide positive attachment. Would weaken the handle a little bit from the holes. In the end I decided to just go with the JB Weld until it proved not up to the task.