May 22, 2011 - Technology media outlet marktechpost published a blog post yesterday (May 21) reporting that Meta The company launched J1 series of models to significantly improve judgment model accuracy and fairness through reinforcement learning and synthetic data training.
Project Background
Large Language Models (LLMs) are breaking out of their traditional roles and gradually taking on the important role of evaluation and judgment. This "LLM-as-a-Judge" model allows AI models to review the outputs of other language models, becoming an important tool for reinforcement learning, benchmarking, and system alignment.
Unlike traditional reward models that score directly, judgment models simulate human thinking through internal chain-of-thought reasoning, which is particularly suited to complex tasks such as mathematical problem solving, ethical reasoning, and interpreting user intent, as well as validating responses across languages and domains, driving automation and scalability in language model development.
However, the challenges faced by the "LLM-as-a-Judge" model are poor consistency and insufficient depth of reasoning, with many systems relying on basic metrics or static annotations, which are not able to effectively evaluate subjective or open-ended questions; another problem is position bias, where the order of answers often affects the final judgment, compromising fairness. Another problem is that position bias in the order of answers often affects the final judgment, compromising fairness.
In addition, collecting manually labeled data on a large scale is costly and time-consuming, limiting the ability to generalize the model.Existing solutions such as EvalPlanner and DeepSeek-GRM rely on manually labeled or rigid training patterns with limited adaptability.
Innovative breakthroughs in the J1 model
To address these issues, Meta's GenAI and FAIR teams developed the J1 model. trained with a reinforcement learning framework, J1 uses verifiable reward signal learning to construct a dataset using 22,000 synthetic preference pairs (including 17,000 WildChat corpus and 5,000 math queries).The models J1-Llama-8B and J1-Llama-70B were trained.
The team also introduces the Group Relative Policy Optimization (GRPO) algorithm to simplify the training process and eliminate positional bias through position-agnostic learning and consistency reward mechanisms.
J1 supports a variety of judgment formats, including pairwise judgment, scoring, and individual scoring, showing great flexibility and versatility.
The test results show that the J1 model is substantially ahead in performance. In the PPE benchmark test, J1-Llama-70B achieves an accuracy of 69.6%, outperforming DeepSeek-GRM-27B (67.2%) and EvalPlanner-Llama-70B (65.6%); even the smaller J1-Llama-8B, with a 62.2% Even the smaller J1-Llama-8B beat EvalPlanner-Llama-8B (55.5%).
J1 also demonstrates top performance in several benchmarks such as RewardBench and JudgeBench, proving its strong generalization ability on verifiable and subjective tasks, showing that inference quality, not data volume, is the key to judging the accuracy of a model.