Not "wrong" ... just not "right".
There's a whole laundry list of possibilities as to why the PRB's didn't group, anything from something you did or didn't did to something mechanical wrong with the gun. Mini's are somewhat less dependent on the crown, muzzle section of the bore and other such things being close to perfect than PRB's do. Did you wipe between shots? Use proper loading techniques? Low-quality balls? Patches not right? yadda, yadda, yadda .......
You really have to begin at square one verifying the parameters of the gun and the quality/consistency of the consumables - that's not a dig on anything/anyone it just is what it is. The rifling for mini's is generally on the fast side for PRB's and pushing a less-than-perfect PRB with too many RPM's makes them go wild. If the nipple is washed out too much, just like a vent on a flintlock, it'll cause very erratic performance being that PRB's are considerably lighter than mini's, the effects are much more easily noticed. Get a notebook and write down everything you try and the subsequent results - start at some point, try different patch/ball combinations, different powder charges and granulations and see what happens. Some guns will shoot about anything, others are very particular on the load parameters and it may take a while to find out what the gun wants. It's not uncommon to have a gun that won't shoot 3F but will group well with 2F; some want a patch/ball combo you have to start with a mallet, others are happy with a combo that's just a little snug.