Whether language models, large language models, or foundation models, transformers require significant computation for pre-training, fine-tuning, and inference. To assist developers and organizations get probably the most performance bang for his or her infrastructure bucks, Hugging Face has long been working with hardware corporations to leverage acceleration features present on their respective chips.
Today, we’re blissful to announce that AMD has officially joined our Hardware Partner Program. Our CEO Clement Delangue gave a keynote at AMD’s Data Center and AI Technology Premiere in San Francisco to launch this exciting latest collaboration.
AMD and Hugging Face work together to deliver state-of-the-art transformer performance on AMD CPUs and GPUs. This partnership is superb news for the Hugging Face community at large, which can soon profit from the newest AMD platforms for training and inference.
The collection of deep learning hardware has been limited for years, and costs and provide are growing concerns. This latest partnership will do greater than match the competition and help alleviate market dynamics: it also needs to set latest cost-performance standards.
Supported hardware platforms
On the GPU side, AMD and Hugging Face will first collaborate on the enterprise-grade Instinct MI2xx and MI3xx families, then on the customer-grade Radeon Navi3x family. In initial testing, AMD recently reported that the MI250 trains BERT-Large 1.2x faster and GPT2-Large 1.4x faster than its direct competitor.
On the CPU side, the 2 corporations will work on optimizing inference for each the client Ryzen and server EPYC CPUs. As discussed in several previous posts, CPUs may be a superb option for transformer inference, especially with model compression techniques like quantization.
Lastly, the collaboration will include the Alveo V70 AI accelerator, which may deliver incredible performance with lower power requirements.
Supported model architectures and frameworks
We intend to support state-of-the-art transformer architectures for natural language processing, computer vision, and speech, corresponding to BERT, DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2. In fact, generative AI models will likely be available too (e.g., GPT2, GPT-NeoX, T5, OPT, LLaMA), including our own BLOOM and StarCoder models. Lastly, we may also support more traditional computer vision models, like ResNet and ResNext, and deep learning suggestion models, a primary for us.
We’ll do our greatest to check and validate these models for PyTorch, TensorFlow, and ONNX Runtime for the above platforms. Please keep in mind that not all models could also be available for training and inference for all frameworks or all hardware platforms.
The road ahead
Our initial focus will likely be ensuring the models most significant to our community work great out of the box on AMD platforms. We’ll work closely with the AMD engineering team to optimize key models to deliver optimal performance due to the newest AMD hardware and software features. We’ll integrate the AMD ROCm SDK seamlessly in our open-source libraries, starting with the transformers library.
Along the best way, we’ll undoubtedly discover opportunities to optimize training and inference further, and we’ll work closely with AMD to determine where to best invest moving forward through this partnership. We expect this work to steer to a brand new Optimum library dedicated to AMD platforms to assist Hugging Face users leverage them with minimal code changes, if any.
Conclusion
We’re excited to work with a world-class hardware company like AMD. Open-source means the liberty to construct from a big selection of software and hardware solutions. Because of this partnership, Hugging Face users will soon have latest hardware platforms for training and inference with excellent cost-performance advantages. Within the meantime, be happy to go to the AMD page on the Hugging Face hub. Stay tuned!
This post is 100% ChatGPT-free.
