James Ding Jul 18, 2024 15:14
Mistral AI and NVIDIA unveil Mistral NeMo 12B, a customizable and deployable enterprise AI model for chatbots, multilingual tasks, coding, and summarization.
Mistral AI and NVIDIA have launched a groundbreaking language model, Mistral NeMo 12B, designed to be easily customizable and deployable for enterprise applications. This model supports a variety of tasks, including chatbots, multilingual processing, coding, and summarization, according to blogs.nvidia.com.
High-Performance Collaboration
The Mistral NeMo 12B leverages Mistral AI’s data training expertise combined with NVIDIA’s optimized hardware and software ecosystem. Guillaume Lample, cofounder and chief scientist of Mistral AI, emphasized the significance of this collaboration, noting the model's unprecedented accuracy, flexibility, and efficiency, bolstered by NVIDIA AI Enterprise deployment.
Trained on the NVIDIA DGX Cloud AI platform, the Mistral NeMo model benefits from scalable access to the latest NVIDIA architecture. The use of NVIDIA TensorRT-LLM for accelerated inference performance and the NVIDIA NeMo development platform for building custom generative AI models further enhances its capabilities.
Advanced Features and Capabilities
The Mistral NeMo 12B excels in multi-turn conversations, math, common sense reasoning, world knowledge, and coding. With a 128K context length, it processes extensive and complex information coherently, ensuring contextually relevant outputs. Released under the Apache 2.0 license, the model encourages innovation within the AI community.
This 12-billion-parameter model uses the FP8 data format for model inference, reducing memory size and speeding deployment without compromising accuracy. Packaged as an NVIDIA NIM inference microservice, it offers performance-optimized inference with NVIDIA TensorRT-LLM engines, facilitating easy deployment across various platforms.
Enterprise-Grade Deployment
The Mistral NeMo NIM can be deployed in minutes, providing enhanced flexibility for diverse applications. It features enterprise-grade software, including dedicated feature branches, rigorous validation processes, and robust security and support. The model is designed to fit on the memory of a single NVIDIA L40S, NVIDIA GeForce RTX 4090, or NVIDIA RTX 4500 GPU, ensuring high efficiency, low compute cost, and enhanced security and privacy.
Optimized Training and Inference
Combining the expertise of Mistral AI and NVIDIA engineers, the Mistral NeMo model benefits from optimized training and inference processes. Trained with Mistral AI’s expertise in multilinguality, coding, and multi-turn content, the model utilizes NVIDIA’s full stack for accelerated training. It employs efficient model parallelism techniques, scalability, and mixed precision with Megatron-LM, part of NVIDIA NeMo.
The training process involved 3,072 H100 80GB Tensor Core GPUs on DGX Cloud, utilizing NVIDIA AI architecture to enhance training efficiency.
Availability and Deployment
The Mistral NeMo model is available for deployment across various platforms, including cloud, data center, or RTX workstations. Enterprises can experience Mistral NeMo as an NVIDIA NIM via ai.nvidia.com, with a downloadable NIM version expected soon.
Image source: Shutterstock