"Intel's Neural-Chat-7B has set a new benchmark on Vectara's hallucination leaderboard, as evaluated using Vectara's open source Hughes Hallucination Evaluation Model (HHEM)—the number one hallucination detection model on Hugging Face* with over 120k downloads. This result underscores three industry trends that are crucial for broader enterprise adoption of generative AI: Firstly, as research in the field has intensified, factual consistency of LLMs has shown steady improvement. Secondly, that the improvement is not limited to proprietary models with API-only access, but also to models that can be freely downloaded and used by the community. Third, smaller, easier-to-deploy models in the 7B and 13B sizes are quite capable, with the right training methods, of achieving strong results on factual consistency. HHEM, developed by Vectara and available to the community, is being increasingly adopted by leading research groups at companies like Intel as the tool of choice for quantifying hallucinations in LLMs."
— Amin Ahmad, CTO and founder, Vectara*