A new assessment by Tsinghua University in Beijing has ranked Baidu’s Ernie Bot 4.0 and start-up Zhipu AI’s GLM-4 as top Chinese large language models (LLMs). However, the report also found that foreign rivals still hold an edge in overall capabilities.
The SuperBench assessment report evaluated 14 representative LLMs, including well-known models like OpenAI’s GPT-4 and Anthropic’s Claude-3. Researchers found that these overseas models outperformed their Chinese counterparts in multiple areas, including semantic comprehension, coding abilities, and alignment with human commands. Notably, the report identified gaps in the code-writing and operative abilities of domestic models, particularly in real-world scenarios.
The assessment aims to provide an objective evaluation framework for the growing number of LLMs emerging in China. The findings serve as a valuable benchmark for Chinese AI companies and researchers seeking to enhance the capabilities of their LLMs.
In recent years, Chinese tech giants and start-ups have engaged in a race to develop LLMs, fueled by the success of OpenAI’s ChatGPT and other generative AI tools. To date, approximately 200 LLMs have been introduced in China, despite the unavailability of OpenAI’s services.
Zhipu AI, founded in 2019, has secured significant funding from various sources, including state-backed investors, venture capitalists, and tech giants like Alibaba and Meituan. Similarly, Beijing-based Moonshot AI recently raised $1 billion in a funding round.
