I usually put the follower in when the tube is down to 1/2 or less, I am also using Dillon tubes. Mine is old enough the tubes are aluminum with the end you pickup the primers with is just slit and pinched in to hold the primers as you pick them up.
The Dillon tubes with the molded plastic pickup ends I drop less primers, tension with the all aluminum tubes can be a bit touchy.
The Dillon tubes also hold 100 primers instead of 50 for the RCBS tubes.
The last 4-5-6 primers don't seem to want feed from Dillon tubes in an RCBS tool as smoothly as the RCBS tubes in an RCBS tool, the little bit of extra weight make them feed right until you run out of primers.