Why does it seem strange?
Within the human race there are hundreds or maybe millions of gods. It is a guaranteed fact that if you are a believer in a single god (monotheism) you do not believe in any of the rest of them. An atheist simply believes in one less god than you. So to most of the universe your one god belief means you are an atheist to all gods except one.
However strange and goofy you think atheists, Mormons, Muslims, Hindus,etc. are you have to know that they think your beliefs are just as unusual. Even more telling is the fact that most of you believers only adopt the faith of your parents and culture. That is not a real logical and error free process.