That just has never been really true. Final as cast size will vary with alloy & casting temp. Sure, use the exact alloy & exact temp, you might get the a bullet that drops the preferred size but not likely. I never saw where Lee molds were regulated with CWW. Most manf use Lyman #2 because its repeatable.
Did you ever actually measure the final bullets? Yes you can certainly get by not sizing, but like everything else in life, it Worls until it doesn't. I size because I load in mixed brass & a fat bullet & thicker case is going to cause an issue. If they are all the same size & there is a problem, I know its the case. One reason I now segregate all my 9mm brass. Thickness is just all over the place & with longer 147gr bullets, I need certain headstamps to make reliable ammo.