A second relevant factor when inhaling large amounts of butane is the quality of the tane itself. All tane manufacturers do include some mercaptans in their butane, giving it a noticeable smell and taste. With the very highest end butane (Vector, Lava, few others maybe) the amount of mercaptan is rather insignificant, leading to a very true taste, but in cheaper butanes (ronson, dodo, etc) there is a large amount of contaminant. This can be observed through the "mirror test," taking a mirror or glass plate and spraying about a dime sized drop of butane onto it, then allowing the butane to evaporate and observing the left over solids. Vector will leave a barely noticeable white mark, while Ronson will leave a dark black one. When smoking with, or inhaling for any reason at all, this less clean butane, the contaminants will be inhaled. These will have a taste.
Lighters bought pre-filled with butane will invariably be filled with the cheapest of the cheap. This does directly lead to many of these style gadgets having an "off" taste, right out of the box.
For an experiment, first purge the tank completely (just press on the filling valve with a pen or something, outside), then refill with high quality butane. In fact, even observing the tip of the device used to activate the filling valve while emptying the tank, may show you some telling signs of the quality of the original tane.