Yep.
A good friend loved heavy rifles. The lightest loads he considered were full bore .45-70. He had a double, not sure if it was a .500 or .577 Nitro Express, that he thought was a great gun to plink with. He and his brother owned many such rifles, bolt, single shot and the big double. They also owned the land for a 600yd shooting range and frequently shot those rifles at the longer ranges.
He was about 5'5" and weighed in at around 150lb. Retired attack pilot (F-105 to the F-111).
PS The other side of the coin is that extended shooting with heavy recoil can have an effect on the body. Taffin was a good example for heavy pistol shooting. Some of the military snipers have had shoulder troubles after prolonged shooting of the heavier calibers.