Yes, you are putting the herb in the wrong end; you should loosely pack the herb into the mouthpiece half of the pipe so it sits on the screen, it shouldn't touch the gray ceramic filter.
It is possible that it isn't ground up enough, but you really don't need it to be very fine to get vapor...