I don't want to shoot at all any more without hearing protection, even .22s, though I do miss going for a walk in the woods on the farm with a .22 rifle and plinking. I don't see a few rounds out doors with a .22 rifle as being damaging.
Due to my years of safety training at work, I wear gloves working in the yard or garden, and ear plug mowing the lawn or running a chainsaw. When shooting larger caliber rifles I've noticed that my ears will ring sometimes with just earmuffs, so I'm getting into the habit of doubling up with plugs too.