I can relate! 'noooo, I cant possibly, I will surely die of exposure'🥶
I have a security blanket in the form of a very light waterproof jacket which folds up tiny, I think of it as my mithril coat, it gets me outta the door!
good call! with me it's hands and feet, mostly feet, I tried neoprene...