I initially felt like you however, no one really cares or notices if you are discrete about it, i'll sit at a café with my lady and fire up a cap by my lap casually, pass it off like a cig. no biggy. perhaps DV's coveted Coffee mug can assuage these concerns with vapers concerned with discretion.
I've never been worried so much about the butane looking bad as the smell. I've yet to use any vaporizer with flower that could be considered discrete as it relates to smell. But you can be discrete in how you do things. I smoked discretely for years around family, friends, fans before concerts. Being discrete doesn't mean people don't know, or notice, it means that you're not making a big deal of it, and as a result neither do they.
When we, we=humans, do things in a way that looks normal and not a big deal, it's typically not seen as a big deal. When we look furtive, or act like we're doing something wrong, people are going to really notice that, and get either concerned, angry, have some other negative reaction.