ADeploy AI Models at Scale to any Cloud

The field of AI is in the midst of a revolution. In recent years, AI models have turned simple text prompts into images, songs, and even his website. With billions of parameters, called base models, these models can be transferred from one task to another with a little tweaking, allowing countless hours of training, labeling, and redesigning models for new tasks. Can eliminate Reconstruction. Artificial Intelligence and Big Data possess a synergistic relationship. Big data analytics leverages artificial intelligence for the best data analysis. 

AI applications are very important as big data analysis tools because they can recognize patterns and provide cognitive capabilities for big data. By using AI with big data analytics, businesses can extract more valuable information from data and gain a long-term competitive advantage.

 The foundation model is primarily trained on high-end, high-performance computing (HPC) infrastructure. This is a reliable but expensive barrier to entry for many who want to train foundation models for their purposes. These systems for training AI models have to be custom-developed and rarely rely on off-the-shelf hardware options. A best-in-class GPU is combined with a low-latency InfiniBand networking system. These systems are costly to set up and operate and require customized operational processes, increasing costs.

Cutting the HPC Cable 

 Researchers collaborate with distributed teams within PyTorch, an open-source machine learning platform from the Linux Foundation, to train large-scale AI models on affordable network hardware found a way to do it. The group's research shows that large models can be scaled and trained using regular Ethernet-based networks on Red Hat's OpenShift platform.

 Using PyTorch's FSDP, the team successfully trained a model with 11 billion parameters using a standard Ethernet network on Cloud. Our approach achieves training models of this size on par with high-performance computing (HPC) network systems, making the HPC network infrastructure effectively redundant for small- and medium-sized AI models.

Allocating Memory to Improve Performance

Earlier attempts to train models with billions of parameters on PyTorch with Ethernet resulted in sub-standard performance, far below what you would require to train a basic model. With cloud computing, systems await to be entirely allocated at all times. As AI models scale, standard methods for data-parallel training only work if the GPU can accommodate a complete replica of the model and its training state. 

While updated training techniques—such as PyTorch's Fully Sharded Data Parallel (FSDP) or DeepSpeed—can effectively distribute the model and data across multiple GPUs during training, they have only worked effectively on HPC systems, not on Ethernet-connected systems. The joint team explored the FSDP API and created a new control called a rate-limiter that controls how much memory is allocated for sending and receiving tensors, easing memory pressure on the system and improving scaling efficiency by 4.5 times over previous approaches.

0