World's first test! OpenAI open source SWELancer, the big model hit 1 million annual salary!

OpenAI launched the SWE-Lancer benchmarking test, selected more than 1,400 real software engineering tasks from the Upwork platform, with a total value of $1 million, and used end-to-end tests to assess AI programming capacity; tests included two broad categories of individual contributors and software managers, namely, assessing code achievement capabilities, while the latter tested technical judgement and decision-making capabilities; Claude 3.5 Sonnet performed best in all test models and earned “over $400,000” on a complete data set, but there was still considerable room for improvement。