- Published on
Meta Llama 3 is as open-source large language model available in two sizes, 8B and 70B parameters. It has been fine-tuned for instruction following, making it more steerable and capable of performing complex tasks.
The model has been trained on over 15 trillion tokens of publicly available data and has demonstrated improved performance on a wide range of industry benchmarks.The model’s architecture has been designed with simplicity and efficiency in mind, using a standard decoder-only transformer architecture with a tokenizer that encodes language more efficiently. The model has also been optimized for inference efficiency, making it suitable for deployment on a wide range of devices.
To ensure responsible development and deployment of the model, Meta has adopted a system-level approach to responsibility, which includes red-teaming (testing) for safety, developing new trust and safety tools such as Llama Guard 2 and Code Shield.
The company plans to release additional models with new capabilities, including multimodality, longer context windows, and stronger overall capabilities, in the coming months.Overall, Meta Llama 3 represents a significant advancement in language model technology and has the potential to enable a wide range of applications and use cases across industries.