Squeeze Beats (CEO Kim Hyeong-jun), a specialist in artificial intelligence (AI) lightweighting and optimization, announced on the third that it has launched ‘Matches on Chips’, a customized solution for serving large language models (LLM).
Matches on Chips is an answer that simplifies the whole LLM serving process and helps find the optimal settings. It’s explained that it provides one-stop support for all steps obligatory for serving LLM, from model selection to adjusting serving options, device and framework settings, performance evaluation, and distribution.
By linking with open source libraries resembling Hugging Face, you’ll be able to easily integrate various LLM models and compare and evaluate performance. It was explained that the prevailing working time of engineers can be reduced by one-tenth, from 30 hours to three hours. He added that costs could be saved by greater than two times.
It also provides a function to match and analyze frameworks resembling vLLM and TensorRT-LLM from various angles. Users can construct an optimized LLM serving environment based on the evaluation results and maximize the general efficiency of the infrastructure. In the long run, we plan to expand the model, hardware, and server environment by linking AI agents or supporting hardware resembling AMD, Amazon, and Google.
Recently, cooperation with Intel and Naver is in progress. We’re working on a collaborative project to efficiently operate LLM on Intel Gaudi hardware. The range of support for Matches on Chips has been expanded to incorporate NVIDIA GPUs and Intel Gaudi. We plan to proceed to support comparison of assorted hardware by way of cost and speed.
Kim Hyeong-jun, CEO of Squeeze Beats, said, “We designed and developed the product in order that anyone can easily simulate and analyze LLM serving,” and added, “We’ll proceed to support AI service corporations in optimizing performance and reducing costs through the event of assorted technologies, including LLM serving solutions.” He said.
Reporter Jang Se-min semim99@aitimes.com
