China’s search engine pioneer unveils open source large language model to rival OpenAI

-

In February, Sogou founder Wang Xiaochuan said on Weibo that “China needs its own OpenAI.” The Chinese entrepreneur is now inching closer to his dream as his nascent startup Baichuan Intelligence rolled out its next-generation large language model Baichuan-13B today.

Baichuan is being touted as considered one of China’s most promising LLM developers, because of its founder’s storied past as a pc science prodigy from Tsinghua University and founding the search engine provider Sogou, which was later acquired by Tencent.

Wang stepped down from Sogou in late 2021. As ChatGPT took the world by storm, the entrepreneur launched Baichuan in April and quickly pocketed $50 million in financing from a gaggle of angel investors.

Like other homegrown LLMs of China, Baichuan, a 13 billion-parameter model based on the Transformer architecture (which also undergirds GPT), is trained on Chinese and English data. (Parameters confer with variables that the model uses to generate and analyze text.) The model is open source and optimized for industrial application, in accordance with its GitHub page.

Baichuan-13 is trained on 1.4 trillion tokens. Compared, Meta’s LLaMa uses 1 trillion tokens in its 13 billion-parameter model. Wang previously said in an interview that his startup was on the right track to release a large-scale model comparable to OpenAI’s GPT-3.5 by the top of this yr.

Having began only three months ago, Baichuan has already achieved a notable speed of development. By the top of April, the team had grown to 50 people, and in June, it rolled out its first LLM, the pre-training model Baichuan-7B which boasts 7 billion parameters.

Now, the foundational model Baichuan-13B is offered without spending a dime to academics and developers who’ve received official approval to make use of it for industrial purposes. Importantly, within the age of U.S. AI chip sanctions on China, the model offers variations that may run on consumer-grade hardware, including Nvidia’s 3090 graphic cards.

Other Chinese firms which have invested heavily in large language models include the search engine giant Baidu; Zhipu.ai, a by-product of Tsinghua University led by Professor Tang Jie; in addition to the research institute IDEA led by Harry Shum, who co-founded Microsoft Research Asia.

China’s large language models are rapidly emerging because the country prepares to implement among the world’s most stringent AI regulations. As reported by the Financial Times, China is predicted to attract up regulations for generative AI with a selected deal with content, indicating more stepped-up control than the principles introduced in April. Corporations may have to obtain a license before launching large language models, which could decelerate China’s efforts to compete with the U.S. within the nascent industry.

ASK DUKE

What are your thoughts on this topic?
Let us know in the comments below.

0 0 votes
Article Rating
guest
0 Comments
Inline Feedbacks
View all comments

Share this article

Recent posts

0
Would love your thoughts, please comment.x
()
x