An AI Odyssey, Part 3: Lost Needle in the Haystack
While shopping on a major e-commerce site, I wanted to get an answer to an obscure question about a certain product.
Not finding the answer immediately on the product page, I thought I’d try clicking the AI shopping assistant helper tool to ask this question.
I waited with anticipation for an answer I would expect be more informative and useful than a standard search result. But it was not to be. The AI tool had nothing worthwhile.
Then I decided on an old-fashioned keyword search across all the product reviews. And, lo and behold, I immediately found several credible reviews addressing my question.
It is not good usability when multiple search mechanisms exist but only one of them is reliable. And it is surprising that a retrieval-based approach (e.g., RAG) could not at least match the effectiveness of a simple keyword search over reviews.
Models are capable, but effective integration can be lacking. Without improvements for cases like this, customers will not be satisfied users of these new AI tools.
Related posts
The post An AI Odyssey, Part 3: Lost Needle in the Haystack first appeared on John D. Cook.
