- Published on
Snowflake AI Research has announced Arctic, a new open source large language model that achieves top-tier performance on enterprise tasks like coding, SQL generation, and instruction following at a very low training cost of under $2 million.
Arctic combines a dense transformer model with a residual mixture-of-experts component to enable efficient training and inference.
It sets a new baseline for cost-effective training of high-quality custom LLMs for enterprises.
The model weights, code, data recipes, and research insights are being fully open sourced under an Apache 2.0 license.
Arctic combines a 10B dense transformer model with a residual 128×3.66B MoE MLP resulting in 480B total and 17B active parameters chosen using a top-2 gating. 4K context window. 32K to come.
Arctic is available now on Hugging Face, the NVIDIA API catalog, Replicate, and other model catalogs, with support for Snowflake’s Cortex platform.