The long run of AI processing

Key findings from the report are as follows: 

• More AI is moving to inference and the sting. As AI technology advances, inference—a model’s ability to make predictions based on its training—can now be run closer to users and not only within the cloud. This has advanced the deployment of AI to a spread of various edge devices, including smartphones, cars, and industrial web of things (IIoT). Edge processing reduces the reliance on cloud to supply faster response times and enhanced privacy. Going forward, hardware for on-device AI will only improve in areas like memory capability and energy efficiency. 

• To deliver pervasive AI, organizations are adopting heterogeneous compute. To commercialize the complete panoply of AI use cases, processing and compute have to be performed on the proper hardware. A heterogeneous approach unlocks a solid, adaptable foundation for the deployment and advancement of AI use cases for on a regular basis life, work, and play. It also allows organizations to organize for the longer term of distributed AI in a way that’s reliable, efficient, and secure. But there are numerous trade-offs between cloud and edge computing that require careful consideration based on industry-specific needs. 

• Firms face challenges in managing system complexity and ensuring current architectures can adapt to future needs. Despite progress in microchip architectures, similar to the most recent high-performance CPU architectures optimized for AI, software and tooling each need to enhance to deliver a compute platform that supports pervasive machine learning, generative AI, and recent specializations. Experts stress the importance of developing adaptable architectures that cater to current machine learning demands, while allowing room for technological shifts. The advantages of distributed compute must outweigh the downsides when it comes to complexity across platforms. 

Download the complete report.