I find with my (old) unit, that with the herb finely ground, after shaking, one can easily get the herb back in the trench. If some does stay on the sides, its not a huge deal, nor do I think it will do any damage, though I could be wrong.
As for poking around in it to get it back down, id say...