According to news from September 9, the Chinese Institute of Automation of the Chinese Academy of Sciences, the Chinese Institute of Automation, Li Guoqi and Team Xu Bo, based on the work of publishing a series of original theoretical papers on internal complexity, have recently worked with Metax to build itBrain pulseLarge Model“SpikingBrain-1.0”, completes process training and reasoning on the GPU Calculator Platform, achieves a quantitative increase in the efficiency and speed of large models in super-long sequence reasoning, and demonstrates the ecological feasibility of constructing new non-Transformer models that are controlled by national production. The research team opened the SpikingBrain-1.0-7B model and opened the SpikingBrain-1.0-76B test site, which simultaneously made public the technical report in Chinese/English on a large-scale industrially validated brain pulse model。This is the first global large-scale brain pulse model, resulting in the domesticization of the entire process, and marks an important breakthrough in the country ' s efforts to integrate brain computing with the innovation of the big model。
1AI WAS INFORMED BY AN OFFICIAL PRESENTATION THAT THE R & D TEAM DREW ON COMPLEX WORK MECHANISMS WITHIN THE BRAIN NEURONSLarge model structure approach based on endogenous complexityThe construction of the Large Brain Pulse Model (SpikingBrain-1.0), which theoretically establishes a link between pulsive neurodynamics and linear attention models, reveals that the existing linear focus mechanism is a special and simplified form of tree-brush computing, thus clearly demonstrating a new and viable path to increasing the complexity and performance of the model. The R & D team went on to build and open a new type of brain base model based on pulse neurons, linear (SpikingBrain-1.0-7B) and hybrid linear complexity (SpikingBrain-1.0-76B, Activating Parameters 12B), and developed an efficient training and reasoning framework for the GPU (MetaX Shayun C550) cluster for national production, a Triton sub-bank, model parallel strategies and cluster communications。
According to Li, this result not only represents a major breakthrough in the country ' s overall process of building a large spectro-pulse model structure and a national production calculator, but also provides a more efficient modelling tool for the application of super-long sequences such as more legal, medical and scientific simulations. It will also inspire the theory of the next generation of neuromorphological computations and the design of chips。