README.md

metadata

title: Arabic Function Calling Leaderboard
emoji: 🏆
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: true
license: apache-2.0
tags:
  - arabic
  - function-calling
  - leaderboard
  - llm-evaluation

🏆 Arabic Function Calling Leaderboard

لوحة تقييم استدعاء الدوال بالعربية

Overview

The Arabic Function Calling Leaderboard (AFCL) evaluates Large Language Models on their ability to:

1. Understand Arabic queries (MSA + Dialects)
2. Select appropriate functions from available options
3. Extract correct arguments from Arabic text
4. Handle parallel and complex function calls
5. Detect when no function should be called

Models Evaluated

• Arabic-Native: Jais, ALLaM, SILMA, AceGPT
• Multilingual: Qwen, Llama, Gemma, Mistral, Phi, BLOOMZ, Aya

Dataset

📊 Dataset: HeshamHaroon/Arabic_Function_Calling

• 1,470 total samples across 10 categories
• Simple, Multiple, Parallel, Parallel Multiple
• Irrelevance Detection
• Dialect Handling (Egyptian, Gulf, Levantine)

Evaluation

The leaderboard automatically evaluates models using the HuggingFace Inference API when the Space starts.

Citation

@misc{afcl2024,
    title={Arabic Function Calling Leaderboard},
    author={Hesham Haroon},
    year={2024},
    url={https://huggingface.co/spaces/HeshamHaroon/Arabic-Function-Calling-Leaderboard}
}