Why? Mostly because there are so many 4 letter words associated wit it: cold, snow, wind, etc. Also there is the fact of shorter days.
But to date, the aggravation has not been enough to overcome inertia and seek warmer climes. One can always put on more clothes but there is a limit to how many you can remove. The current NY tax situation, with legislators enacting retroactive taxes may change that.