DeepSeek launches FlashMLA: A breakthrough in AI speed and efficiency for NVIDIA GPUsFollowing the success of its R1 model, Chinese AI startup DeepSeek on Monday unveiled FlashMLA, an open-source Multi-head Latent Attention (MLA) decoding kernel optimized for NVIDIA’s Hopper GPUs. Think of FlashMLA as both a super-efficient translator and a turbo boost for AI models, helping them respond faster in conversations and improving everything from chatbots to voice assistants and AI-driven search tools.
FlashMLA is designed to maximize AI efficiency. It supports BF16 precision, uses a paged KV cache with a 64-block size, and delivers top-tier performance with 3000 GB/s memory bandwidth and 580 TFLOPS on H800 GPUs.
The real magic is in how it handles variable-length sequences. This significantly cuts down computational load while speeding up AI performance—something that has grabbed the attention of AI developers and researchers.
- High Performance: FlashMLA achieves up to 3000 GB/s memory bandwidth and 580 TFLOPS computational throughput on H800 SXM5 GPUs, utilizing CUDA 12.6.
- Optimized for Variable-Length Sequences: Designed to efficiently handle variable-length sequences, enhancing decoding processes in AI applications.
- BF16 Support and Paged KV Caching: Incorporates BF16 precision and a paged key-value cache with a block size of 64, reducing memory overhead during large-scale model inference.
Github: github.com/deepseek-ai/FlashMLA