Open-source AI narrows gap with proprietary leaders, new benchmark reveals

Artificial intelligence startup Galileo released a comprehensive benchmark on Monday revealing that open-source language models are rapidly closing the performance gap with their proprietary counterparts. This shift could reshape the AI landscape, potentially democratizing advanced AI capabilities and accelerating innovation across industries.

The second annual Hallucination Index from Galileo evaluated 22 leading large language models on their tendency to generate inaccurate information. While closed-source models still lead overall, the margin has narrowed significantly in just eight months.

“The huge improvements in open-source models was absolutely incredible to see,” said Vikram Chatterji, co-founder and CEO of Galileo, in an interview with VentureBeat. “Back then [in October 2023] the first five or six were all closed source API models, mostly OpenAI models. Versus now, open source has been closing the gap.”