The HELM Arabic benchmark just gave Arabic AI its first shared public scoreboard — and Dubai is at the centre of it. Arabic.AI has partnered with Stanford University's Center for Research on Foundation Models (CRFM) to launch HELM Arabic, a structured evaluation framework designed specifically for Arabic large language models. For years, Arabic LLM developers worked without a universal benchmark. Claims about model performance circulated widely, but there was no common standard to test them against. Now there is.
The HELM Arabic benchmark brings Stanford's rigorous evaluation methodology into the Arabic language space, creating a public leaderboard where models are assessed under consistent, reproducible conditions. For founders, researchers, and enterprise teams across Dubai and the wider region, this is a defining moment.
What HELM Arabic Actually Means
HELM stands for Holistic Evaluation of Language Models. Developed by Stanford CRFM, the framework evaluates large language models across multiple dimensions — accuracy, reasoning, bias, and robustness. It has been widely adopted for English models and is recognised across academic and industry circles.
Arabic, spoken by more than 400 million people worldwide, had no dedicated extension of this framework. That gap left regional AI teams without a standardised way to measure model performance on Arabic tasks.
HELM Arabic changes that.
The platform introduces structured evaluation tasks tailored to Arabic language understanding and generation. Results are published on a public leaderboard hosted through Stanford's HELM interface. For startups and enterprise teams, this creates real clarity: instead of relying on internal testing claims, developers can now see how models perform under the same academic framework everyone else uses.
The Leaderboard and LLM X
With the first phase of HELM Arabic live, attention turned quickly to the rankings. According to the published leaderboard, Arabic.AI's proprietary model LLM X — also referred to as Pronoia — currently ranks first across seven evaluation task clusters, recording the highest overall performance in the initial release.
Arabic.AI is also the only non-open model on the leaderboard that was trained specifically for Arabic language use. Several open-weight regional models appear on the board — including AceGPT v2, ALLaM, and earlier versions of Jais — but their scores place them below the top tier in this release.
It is worth noting that some of the evaluated model versions date back to 2024. AI development cycles are fast, and updated iterations may perform differently in future benchmark rounds. Global open-source models such as Qwen and Llama have also placed within the top ten, showing strong multilingual performance across Arabic evaluation tasks.
A Statement From Leadership
Nour Al Hassan, CEO of Arabic.AI, stated that Arabic language models have historically received less attention in foundation model research. The collaboration with Stanford CRFM aims to place Arabic AI evaluation on equal academic footing with English benchmarks.
For Dubai's growing AI ecosystem, that matters. Standardised benchmarking strengthens trust in local innovation, supports enterprise adoption, and positions regional AI products within global research conversations.
The HELM Arabic benchmark is now live and the leaderboard is public. For AI founders in Dubai, this sets a new baseline: model performance can be evaluated through a recognised academic framework rather than isolated claims. As more models are tested and updated, the leaderboard will continue to evolve. Dubai's AI scene now has a global reference point — and that changes the conversation.
