A lot of barrels are installed and shipped with some internal stresses remaining as well. Part of many barrel mfg processes is to take a barrel fresh from rifling and check the bore for being straight. If it is not, they bend the barrel until it is. I suspect a barrel that is bent "a lot" will change POI pretty dramatically as it heats up.