Why your AI-review process is going to fail

A lot of companies say they use human-in-the-loop (HITL) to review AI output, and they say it in a way that means, “We’re not like the others.”

Reviewing AI output for plausibility isn’t enough

Even the most gung-ho AI user has to admit that LLM-based systems can, at best, be inventive and, at worst, downright wrong. But are people the best at assessing the correctness of output?

HITL is usually implemented in a very informal way — once a result or piece of content is generated, someone looks at it and issues a ruling on the work’s acceptability. In short, it’s assessed for plausibility. There’s a glaring problem with that: Plausibility is not accuracy.

You can read a thought-piece I wrote on this topic for MarTech Magazine called Why your AI-review process is going to fail.