The question that have always asked my self is this: If the house is on fire what difference is 15 or 20 feet going to make? Powder burns (if not confined) primers exploded and ammo pops when heated.
"I am a bit concerned about the primer tube detonating and igniting the powder and setting off the ammo." You might want to re think the "Hard Hat" comment. Seriously, ammo and components are relatively stable if handled correctly. Thankfully most of use do not live a life like a Final Destination movie.
Prudent and reasonable precautions:
Don't leave powder canisters open on the bench.
Don't Smoke or eat while reloading.
Don't take out more primers then you are going to use.
Wear safety glasses.
If something sticks/jambs do not force it! Stop, Find out why!
Put everything away when you are finished.
Mount an ABC fire extinguisher at the door.
I could continue on, but I think you get the idea.