Super Miniature Big Model

Recently, I have been interested in small-sized AI models. In the past, domestic and foreign technology companies in the field of large models generally continued to increase model parameters to enhance model capabilities, from 700 million, 1.3 billion, 7 billion, and even more than 20 billion. It is indeed getting stronger and stronger, but this will also make everyone's own computer basically unable to run. Recently, Microsoft released the Phi-2 large model with only 270 million parameters, and Alibaba also released the Tongyi Qianwen super mini large model with only 180 million parameters. These small-sized large models not only surpass some large models with 700 million or even 1.3 billion parameters in performance evaluation, but also because of their small size, the basic...