I just tried one (my antenna port is connected to a cable and a longer antenna with gain). I do agree that finger tight is not enough to get it to stay vertical without it flopping down pretty easily. I would not overtighten it however. It will work just find horizontal as long as it't not laying on a metal surface.
One thing, if you remove it, is the socket tight in the base...can't rotate it with your fingers. Then it is normal and just the way it is. Those SMA connectors are not usually meant for holding an antenna in place but rather for connecting a transmission line. If it were for holding an antenna in place, they would have a serrated surface that locked into the plug end when snugged up instead of the smooth surface it does have.