Tencent AI Lab, SSE unveil overthinking of o1-like mega-language models

Large-scale language models of class o1 suffer from the phenomenon of "overthinking", which consumes too much computational resources on simple problems and leads to inefficiency; the paper proposes a new efficiency index to evaluate the accuracy and diversity of o1 models in the reasoning process, and finds that extending the chain of thinking does not significantly improve the accuracy; through self-training and other optimization strategies, we can reduce overthinking and maintain accuracy while effectively reducing computational overhead. overthinking and maintain accuracy while effectively reducing computational overhead through optimization strategies such as self-training.