Meta has introduces its first open source "frontier level" Llama 3.1 405B model, which it claimed will have better cost and performance relative to a closed model.
The company on Tuesday also released "new and improved" versions of Llama 3.1 70B and 8B models, built with 70 billion and 8 billion parameters.
"The models will be available on all major clouds including AWS, Azure, Google, Oracle, and more. Companies like Scale.AI, Dell, Deloitte, and others are ready to help enterprises adopt Llama and train custom models with their own data," Meta Platforms CEO Mark Zuckerberg said in a blog post.
In 2023, Llama 2 was only comparable to the models of the older generation, and this year, Llama 3 is competitive with the most advanced models and is leading, he said.
Llama 3.1 is a more complex model than Llama 3, with 405 billion parameters. It has been trained with over 16,000 of Nvidia’s ultra-expensive H100 GPUs.
With a longer context window of 128k, the latest model can understand bigger paragraphs, even entire research papers, and summarise them in smaller bits of key information.
The new model can converse in eight languages, write higher-quality computer code and solve more complex math problems than previous versions, the company said.
"I believe the Llama 3.1 release will be an inflection point in the industry where most developers begin to primarily use open source," Zuckerberg said.
Meta is working with other companies, including Microsoft, Amazon, Google, Nvidia, and Databricks, to help developers deploy their own versions of Llama 3.1. The company also claims that the launched model costs roughly half that of OpenAI’s GPT-4o to run in production.
Zuckerberg said he made the model open source because of his experiences in his formative years with Apple
"Between the way they tax developers, the arbitrary rules they apply, and all the product innovations they block from shipping, it’s clear that Meta and many other companies would be freed up to build much better services…" he said.
Image Source: Unsplash