benchmarking

April 14, 2011 - LMArena has updated the rankings of Meta's newly released open-source big model, Llama-4-Maverick, and it has plummeted to 32nd place from its previous position of 2nd. This confirms the developers' suspicion that Meta provided LMArena with a "special edition" of the Llama 4 model in order to brush up the rankings. On April 6, Meta released its newest big model, Llama 4, which comes in three versions: Scout, Maverick, and Behemoth. The ...

As artificial intelligence (AI) technology continues to evolve, so-called "reasoning" AI models have become a hot research topic. These models are able to think step-by-step like humans and are considered more capable than non-reasoning models in specific domains, such as physics. However, this advantage comes with high testing costs, making it difficult to independently validate the capabilities of these models. According to data from Artificial Analysis, a third-party AI testing organization, evaluating OpenAI's o1 inference model against seven popular AI-based...

MLCommons, the open machine learning engineering consortium, yesterday announced the release of version 0.5 of the MLPerf Client benchmark for measuring AI performance on consumer PCs, the first public version of the test. MLCommons said the MLPerf Client benchmark is the result of a collaborative effort by stakeholders such as AMD, Intel, Microsoft, NVIDIA, Qualcomm, and top PC OEMs, all of whom contributed their expertise and resources to the test. MLPe...

UL Solution, the developer of 3DMark, announced the launch of the Procyon AI Text Generation Benchmark on September 9, local time, which comprehensively judges the text generation capabilities of AI gas pedal hardware by using a wide range of large-language AI models with different parameter scales. The Procyon AI Text Generation Benchmark currently supports local NVIDIA, AMD, and Intel GPUs via the DirectML Common API, as well as Intel's own GPUs via Intel's OpenVINO (Note: discrete and integrated graphics...