Pardon me for chiming in but let me throw my 2 cents into the pot.
In the state of Texas you can ask ANY game warden who knows his trade and he will tell you that even though it is ILLEGAL more deer have been taken in the history of the state with a 22 long rifle than ANY other cartridge. I am not suggesting you do that please keep reading. In my family on of my great grandfather hunted ONLY with a 32-20 with factory lead loads, I never knew him not to drop a deer with a single shot at any given range he felt comfortable shooting at. This is also a round that many do not suggest you use. I have seen the opinions the OP spoke of on OTHER boards. Haven't seen them on this board. What it boils down to is shot placement on the critter. IF you can make a clean kill with a given caliber, AND you are comfortable doing so at X range then by ALL means proceed. The REASON so many people suggest heavier faster heavier faster heavier faster is because many people (not necessarily the ones suggesting it) can't shoot WORTH BEANS! I have seen a cousin of mine bang away at a deer with every round from a 30-30 at 50 yards and he didn't even graze the critter. I have seen my grandfather (one of 5 sons and 2 daughters of the great grandfather mentioned above) drop a deer with a single shot from an iron sighted 222 remington at more than 200 yards. It is a shot I would NEVER try to make as I know I can't do it, but this man was a Korean War Marine vet (E25) who survived the Chosin Reservoir trap and if he could see it it was dead meat in the freezer and on the grill every time with a single shot. He took 60+ deer with 60+ matching clean shots in my lifetime alone, and NO round was EVER fired from his gun unless it was fired to take a critter. He just didn't miss. Ever. Sadly he was taken by heart failure eight years back, but he lived a good long life, even if it was a hard one.
My point is that when it is all said and done the shot you are comfortable that you can make a clean and effective kill with is the shot you should take, and NO other shot should be taken. When recommendations are made by folks to the contrary it is usually because the shot isn't one the general public should ever try because they just wind up with wounded animals, dead cows, and Lord knows what other property damage. The flip side to that coin is the innumerable self appointed experts who read what innumerable self appointed experts wrote at some point and regurgitated the malarkey word for word. When it comes down to it there are three major factors IMHO for a shot (keep in mind I am a admitted rank amateur). #1 above and beyond all else is can YOU the shooter make the shot? #2 Can the firearm in question be counted on to make the shot (you should know if your gun shoots a five foot group at 100 yards from a bench rest that you shouldn't take a 100 yard shot on ANYTHING) and #3 is can the round/caliber do the job with the shot you are about to make? #3 is the MOST debated rule of the shot, and where some folks come up with the idea that only X or Y will work and W or Z can't possibly work. Much of which is old wives tales, whole cloth, and hogwash mixed with a smidgen of fact to make it palatable.
I have taken deer with 45 long colt cowboy loads at ranges that nobody would believe if I posted it and seen the round do a full penetration on the critter. I have seen people who couldn't take a deer at 50 yards with a scoped 30-06/7mm Mag/etc to save their life. Sometimes I look at the upwind critter and decide that maybe today I can't make that shot. I'm not feeling solid at that range so I either get closer or don't shoot.
In the end you have to make the choice, and only YOU the shooter can decide because only YOU the shooter have to live with what happens when that trigger breaks over and that powder goes off.
GoodOlBoy