I had an extra house for about a year. I was about an hour's drive. Good hunting and fishing but having two houses drove me crazy. It made me mean. People had begun driving their 4 wheelers through the yard.
I still think about getting a "camp". The perfect situation would be a lot on the edge of some public land, with electricity. I would put up a garden shed to store some chairs and shovels, etc.