Tsinghua

March 14, 2011 - Professor Zhai Jidong's team at Tsinghua University's Institute of High Performance Computing (IHPC) and Tsinghua-based startup Qingcheng Jizhi jointly announced today that the large model inference engine "Chitu" is now open source. According to the introduction, this engine realizes for the first time to run FP8 accuracy models natively on non-NVIDIA Hopper architecture GPUs and various types of domestic chips, halving the cost and doubling the performance of DeepSeek inference. Positioned as a "production-grade large model inference engine", the engine offers the following features: Multiple arithmetic adaptations: it not only supports NVIDIA's latest flagships to ...

Recently, Tsinghua University and Harbin Institute of Technology jointly published a paper that successfully compressed a large model to 1 bit while maintaining the performance of 83%. This achievement marks a major breakthrough in the field of quantization models. In the past, quantization below 2 bits has always been an insurmountable obstacle for researchers, and this attempt at 1-bit quantization has attracted widespread attention from the academic community at home and abroad. The OneBit method proposed in this study is the first attempt to compress a pre-trained large model to a true 1 bit. Through a new 1-bit layer structure, SVID-based parameter initialization and quantization…

Recently, a series of open-source visual models with excellent performance have emerged under the promotion of China's top universities such as Tsinghua University and Zhejiang University, which are open-source alternatives to GPT-4V. Among them, LLaVA, CogAgent and BakLLaVA are three open-source visual language models that have attracted much attention. LLaVA is a large multimodal model trained end-to-end that combines the visual encoder and Vicuna for general vision and language understanding, and has impressive chat capabilities. CogAgent is an open-source visual language model improved on CogVLM, with 11 billion...