Both kinds of shell holders (the flat Lee primer seater type and the standard bayonet type) work fine in the two generic inertia pullers I've had. The decision about whether or not to use an inertial puller is a different matter from whether to use the 3 piece collet or shell holder in the inertia (impact) puller.
The only downside I've noticed to using a shell holder in place of the 3 piece aluminum collet in the inertia pullers, is that in many of the inertial pullers the shell holder is not held tightly and the unit rattles a bit when whacking it. Otherwise very little difference.
Some confusion may have crept into this thread because of the word "collet". The die/press mounted grip pull collet system is different from the inertia (impact) system that the OP was dealing with. Both use "collets" for difference purposes.