Nvidia's research team has successfully launched Llama-3.1-Minitron4B using model pruning and distillation techniques. This is a compressed version of the Llama3 model, aimed at implementing artificial intelligence on devices. The model reduces the parameter count of the original 8B model through deep and width pruning techniques while maintaining performance close to larger models. Despite a significant reduction in training data (by 40 times), the model achieved a 16% performance improvement on the MMLU benchmark.

Recently, NVIDIA launched the Minitron series of small language models, including 4B and 8B versions, significantly increasing training speed by 40 times while greatly reducing resource and data requirements, resulting in cost savings. By combining techniques of 'pruning' and 'knowledge distillation', the Minitron models maintain performance while reducing size, allowing developers to harness advanced technology for applications such as translation, sentiment analysis, and dialog AI at a lower cost. The open-source nature of the Minitron models enables more people to easily access and use them.