You're right. Quick, go tell the wife......
Notice the treads on the glass part in the photo:
Ignore the rubber pieces, that was a project to replace the lost rubber gasket under the cap (the reason, no doubt, you're having troubles).
They usually come off easier when hot. Don't twist by...