I would check the lighter you use first. I have been using butane vapes exclusively since 2017, I use Colibri, Swan and Newport butane and have never, ever tasted butane whilst vaping. A quality torch lighter will burn all of the butane and leave to trace.
Next thing I would check is the O-rings.