Chinese businessman and founder of Sogou Wang Xiaochuan on Tuesday unveiled a next-generation huge language model Baichuan-13B.
Its founder’s illustrious career as a computer science prodigy from Tsinghua University and founded the search engine provider Sogou, which was later acquired by Tencent, Baichuan is being hailed as one of China’s most potential LLM developers.
At the end of 2021, Wang left Sogou. The businessman founded Baichuan in April and soon collected $50 million in funding from a group of angel investors as ChatGPT grabbed the world by storm.
Baichuan, a 13 billion-parameter model built on the Transformer architecture (which also supports GPT), is trained on data in both Chinese and English like other Chinese-made LLMs.
According to the model’s GitHub page, it is open-source and designed for use in commercial applications.
1.4 trillion tokens are used in Baichuan-13’s training. In contrast, Meta’s LLaMa’s 13 billion-parameter model consumes 1 trillion tokens.
Baichuan has already developed at a noticeable rate. The team grew to 50 members by the end of April, and in June it unveiled its first LLM, the pre-training model Baichuan-7B with 7 billion parameters.
The fundamental model Baichuan-13B is now freely accessible to researchers and programmers who have obtained formal permission to use it for commercial endeavours. The model provides variations that can run on consumer-grade hardware, including Nvidia’s 3090 graphic cards, which is significant in the era of U.S. AI chip restrictions against China.
As China gets ready to enact some of the strictest AI restrictions in the world, huge language models are developing quickly.
