Yes, all stems, or "aroma tubes" as they're called there, can be capped and my caps do fit an AA battery, I just tried!
Those are the tipped ones they are quite nice I have the longer 70mm one, I thought the shorter 60mm one might make things hotter, some love the all glass ones too.
I think you...