MOREH (CEO Jo Kang-won), a specialist in artificial intelligence (AI) infrastructure solutions, has opened its self-developed Korean foundation large language model (LLM) ‘Llama-3-Motif-102B’ to Hugging Face. It was announced on the third that it might be released as.
As will be seen from the model name, it is predicated on the ‘Rama 3.0 70B’ model. Because the parameters have been greatly expanded to 102 billion, it might be seen as a brand new foundation model, and it is alleged that it is especially focused on ‘strengthening Korean answer performance’.
Since all domestic models haven’t revealed their parameters, they can not be compared officially, nevertheless it is alleged that it can be the biggest domestic model. The industry estimates that Naver’s ‘HyperclovaX’ is price 70~80B.
Motif was released in two versions: a pre-trained language model and an instruction model specialized for following instructions.
Lim Jeong-hwan, director of More AI, said, “The model development goal largely consists of three processes.”
First, he said, “This can be a case of maximizing performance throughout the spectrum covered by the prevailing model (Rama), and this has already been achieved by rating first on the Open LLM leaderboard.”
Actually, MoMore introduced the English LLM ‘MoMo-70B’ early this yr and ranked first on this planet, scoring 77.29 points on Hugging Face’s global ‘Open LLM Leaderboard’. On the time, it was explained that this was an achievement achieved in only three months from the beginning of model development to achieving first place.
Director Lim Jeong-hwan said, “Based on the know-how gained during this development process, we were in a position to learn more complex sentences (depth) and complete a motif that creates fluid expressions (width) in conversation.” He added, “That is the second process, ‘existing model’. “It corresponds to ‘producing greater than ideal results,’” he said.
He continued, “It’s true that the Rama 3 70B has a certain level of Korean performance, nevertheless it is way worse than English,” and added, “The last word goal of this open source model is to maximise Korean performance.”
Actually, within the ‘KMMLU’ benchmark, a Korean version of the AI evaluation system, Motif ranked first with 64.74 points. This can be a result that exceeds the 54.5 points of the bottom model, Rama 3 70B, by greater than 10 points.
It also surpassed OpenAI’s ‘GPT-4o (64.11 points)’ and Alibaba’s ‘Q12 (64.1)’, generally known as the strongest open source.
It recorded higher Korean benchmark scores than models similar to LG AI Research Institute, Naver, and Upstage.
This performance was explained by way of an enormous amount of Korean language learning amounting to 187 billion tokens and unique learning techniques. Along with articles that will be collected on the internet, publicly available skilled documents (domestic patents, research reports, etc.) were used as learning data. As well as, the biggest Korean language purification data within the country was secured and included in learning.
Above all, he said, it’s because the corporate has developed and possesses the optimal infrastructure and AI platform technology for AI model development. More’s ‘MoAI’ platform helps you efficiently develop and learn LLM through highly parallel processing techniques.
Actually, Moret is collaborating with global semiconductor company Tenstorrent on software technologies similar to GPU virtualization technology. This goals to transcend NVIDIA’s GPU and CUDA software proprietary structure.
Unlike other latecomers who concentrate on inference, he emphasized that the AI data center solution that mixes Tenstorent NPU and Moret SW will be used widely, not just for inference but in addition for foundation model learning. It has already undergone significant development and is scheduled to be commercialized and launched in the primary half of 2025.
AI Director Lim Jeong-hwan emphasized, “Like this, the day after tomorrow is competing with global big tech.”
He also said, “When it comes to model development, the third and final goal is to create a totally Korean-style foundation LLM from scratch, fairly than basing it on other models.”
To speed up this, we’re also planning to determine a subsidiary. Specifically, he said that ‘development of a multimodal model’ is considered one of the most important goals. Within the case of the inspiration model, we plan to begin small and progressively increase it. Subsequently, it is alleged that a ‘small’ parameter model can also be a really possible future. The goal is to develop an LLM specialized in specialized areas similar to medicine, law, and finance.
Cho Kang-won, CEO of Moret, said, “The rationale we’re releasing the high-performance LLM as open source in order that anyone can use it’s, above all, to assist the domestic AI ecosystem grow in a more progressive direction and contribute to sovereign AI.” “As this can be a very unusual case of disclosure, I hope that many firms will actively put it to use.”
Meanwhile, Moret can also be actively recruiting talent. “We ask on your interest in our journey to determine a foundation model at the extent of world big tech,” he said.
Reporter Jang Se-min semim99@aitimes.com
