Your definitely doing everything right. Sounds like a contact isn't right which isn't a difficult fix but you will need the right torx bit to unscrew the top cap to preform the procedure. In the meantime if you want to do some easy trouble shooting all you need is a pencil eraser, preferably a white one but any eraser that doesn't leave any sort of marking or discolouration when rubbed on paper you should be ok
start by taking off the battery door and give it a good going over on the contact point and the threads then the threads on the tinymight make sure to do that upside down so no eraser bits fall in, while you're at it clean the battery as well. Reassemble and try using it again, if it doesn't help I can walk you through the other procedure