It should stay in place and be very tight normally, did you take off the o-ring and maybe put a smaller one in place?
Make sure that's it's pushed onto the o-ring as far as it can go, you might need to lube the o-ring slightly, when it's completely dry it can be hard to push on. It should align...