The argument that humans make mistakes too so we should be okay with AI making mistakes is… bizarre. I don't use a calculator for it to be wildly wrong some of the time and not indicate any sort of confidence level. I use a calculator when I want the right answer, ALL THE TIME.
Sure, I also don't ask the calculator to write poems in the style of Mad Hatter… but maybe, just maybe, I can do without that.
Also, again, "AI" is not sentient. It's actually not 'making mistakes'. It's just operating according to its programming. Which, people for some reason forget, is entirely human-made. Yes, even when it's probabilistic. "Hallucination" is a misdirection.