I came to that conclusion years ago. When the only difference was the pistol, the smaller pistol won out for concealment. Now that the 9mm pistols are the same size as the .380's, I can no longer go with the .380.
The difference in recoil between a 95 gr .380 and a 115gr 9mm is so small that I wouldn't even consider it to be a factor.
There are some great .380 pistols available and if I had one I was proficient with I'd certainly keep it but I wouldn't go out a buy a new one.