That depends on a few things. However, if they are quality trusted batteries, a battery cap on one end or are in a case, are in a cool dry place, and not in your pocket, they will not spontaneously explode.
I would not carry them around all day loose in a pocket, but in a proper battery case...