Considering that you lack the data collection ability a chronograph gives you, I think five shots is probably the best balance between the false economy of three shots (which won't reliably give you a good sampling between the potential for shooter error and load variability), and ten shots (which will let you weed out called flyers, look for erratic behavior at that load level, and really let you know if you "have something").
I suppose it is feasible to apply the "ladder test" methodology even if you don't have a chronograph. The practice my Dad and I have applied for more than 25 years is to fire a series of single shots over the chrono - each of them containing 0.2 to 0.3 grains more powder than the last. We usually find "Nirvana" for the final load when we get a sequence of 3-5 shots where the velocity doesn't significantly vary, and those same 3-5 shots don't wander about the target. We pin our hopes in the middle of the high and low charges and test it out with about ten matching rounds the next trip to the range. Without the chrono, you'd be making an assumption about stable velocity, and would have to spot and record your impacts carefully, but you would have likely achieved the same result - finding the "dead spot" where a half grain or so of variation powder practically changes nothing downrange.
Might save you some components.