Wiki is loaded with info, paraphrased here with personal opinions/observations:
The Scotsman's bag is a sporran, which seems to be the Scottish name for a purse. People (men and women) carried purses for money (mainly coins) until pants started being made with pockets, which wasn't until the later 1700's. Women also carried their stuff in pockets that were worn around the waist under their dresses [concealed carry fanny packs!]. They then started carrying an "indispensibles" bag [sounds a lot like a "possibles" bag LOL], also called a reticule from the French. Since it was carried by hand the name morphed into a "hand bag". Then, since women carried money in them they started calling them purses as well.
Men carried bags for various reasons as well. Usually shoulder bags. A purse was for money but the bag was for whatever needed to be carried, usually with a strap over the shoulder. Sashes and belts supported weapons and could be used to secure a purse. So a heavy load of ammo would be held in a shoulder bag as well, or on the belt supported by a sash, again a version of the fanny pack or belt pack. Even today the "Sam Browne" is a modern take on the sash and belt of the sword days. So, back then EDC stuff would be carried on the belt or in a bag supported by a sash. When pockets came around then the coin purse was no longer needed so men no longer carried 'purses'. Many also stopped wearing weapons every day and relied on suspenders for the pants and pockets for coins. Bags for papers and other bulkier items were shoulder bags or messenger bags. Even today you can get briefcases with shoulder straps.
Why did men not carry "handbags"? Probably cause of Freud. He assigned a 'sexual' nature to the woman's handbag. I can see how that would result in the universal disdain of the handbag by men.
Now men can still carry stuff, but, they call them packs instead of handbags or purses. Butt pack, fanny pack. Some versions are still called bags, eg, messenger bags, camera bags, etc. But, they are not normally for EDC. Again, pockets and belts take care of EDC stuff. If a man is carrying a fanny pack or shoulder bag it is 'assumed' that they are carrying something other than EDC stuff: papers, computers, lunch, etc, etc.
Looking back on this there is also the old trend in keeping one's hands free from carrying things. The hands had to be free to draw a sword and knife on a moments notice. A handbag would only slow the process and yield valuables to be snatched. So, everything a man carried needed to be secured in pockets, on the belt or in a shoulder bag.