I was looking myself at various pearl/ball materials online the other day and was surprised to read that effects on flavor were an issue with some materials. I don’t remember anything specifically about zirconia balls in that regard, but in discussions of glass/boro, quartz, rubies, titanium, SIC, <some other SIC-like thing>, etc., I recall rubies being mentioned as better flavor than some. There were also r/vaporents and maybe FC (?) discussions of health issues, mostly around breathing in sharp dust particles. No guarantees, but holes drilled in beads (probably intended for craft usage) was considered a bad idea, and even just balls clacking against each other was suspected by some, who claimed to have seen visible dust. OTOH, some people remarked that if you are at the beach or otherwise around wind & sand, you might be getting more of the bad stuff from that (again, don’t trust me or my second-hand interwebs info—I’m neither a micro-geologist nor a pulmonologist or whatever, and I suspect none of these people were either).
Anyway, I decided to hold off on any cooling balls for now, as flavor is a huge issue for me, FWIW / YMMV.