I had started doing the same thing. I would break up the pucks and grind them while the next one was sitting under pressure in the press.
Today I started on a new press die. I am now working in a hydraulic shop so I have access to honed steel tubing. I'm using a piece that is 2.5 inches ID, and making the bottom cap and pusher out of aluminum. It will be considerably lighter than my all steel one I've been using for years and with the nice honed bore should be easier to get the pucks out. Hoping to try it this weekend.