What's it all about? do you have some picture to share with us? what kind of wood are we talking about? what is the configuration?
Is this your first brick-type vaporizer? Personally, I've noticed that if you stir and tamp your bowl often during a session, small herb particles can pass through the screen. That's why I add an extra screen under the mouthpiece glass on all my brick-type vaporizers (because I like to stir and tamp my bowl also smell the goodness of roasted weed). The secondary screen has become primordial for me and if you try, you'll see that small herb particles accumulate quite often in fact (this is especially true if you stir frequently and chop your herb finely enough). Of all my bricks, this is more noticeable on the NOVA because it has a larger bowl diameter than my other bricks (but this problem is quickly solved by adding a second screen). Maybe that's your problem?
To be precise, the flame must never enter the flame intake in its entirety, but must absolutely remain outside (on the rim or a little further away, depending on the power of the torch). You'll also need a good torch lighter (preferably electronic, as mechanical lighters can release particles in the air path caused by the stone). You'll also need to obtain quality butane (i.e. with as few impurities as possible, take the one with the most refining).
Personally, I don't encounter any particular problems with this type of vaporizer, for example, compared to combustion over several years, my health is much better. Coolbreeze sums up the situation very well, perhaps you have an alergy with the essence of the wood or wax?
I often notice users coming from the world of electronic vaporizers indicating that the taste is not the same on a brick type vaporizer, for my part it's quite the opposite, in fact I really started vaping with bricks and I think that's what creates the point of comparison for these people. I wouldn't say their judgment is flawed but we just have different experiences. If you always compare your first experiences with what you know now, your judgment is bound to be affected.
I have many different types of vaporizers and I always come back to my bricks, which I find to be the best performers with a majestic taste (despite the wood). The only exception for me is the CL (and more rarely TM2).