Supported dialects
Detection is marker-based, so its accuracy depends on whether a comment actually contains dialect-specific vocabulary. A short or formally-worded Arabic comment carries no markers and correctly returns
msa with a low confidence. Treat dialect_confidence as the signal for how much to trust the label.Example classifications
Gulf Arabic
Gulf Arabic
Comment: “وش رايكم بالمحتوى الجديد؟ أنا شايف إنه وايد حلو”Translation: “What do you think of the new content? I think it’s very nice”Markers:
وش (what - Gulf), وايد (very - Gulf), حلو (nice - Gulf)Egyptian Arabic
Egyptian Arabic
Comment: “الفيديو ده جامد أوي، عايز تاني كده”Translation: “This video is amazing, I want more like this”Markers:
ده (this - Egyptian), أوي (very - Egyptian), عايز (I want - Egyptian), كده (like this - Egyptian)Levantine Arabic
Levantine Arabic
Comment: “كتير حلو الفيديو، بس شو القصة ورا الأغنية؟”Translation: “Very nice video, but what’s the story behind the song?”Markers:
كتير (very - Levantine), شو (what - Levantine)Modern Standard Arabic (MSA)
Modern Standard Arabic (MSA)
Comment: “يجب أن نشجع هذا النوع من المحتوى الهادف”Translation: “We should encourage this type of meaningful content”Markers:
يجب أن (must - MSA formal), هذا النوع من (this type of - MSA structure)How it works
How the pipeline works
Dialect detection runs inside NAGL (NAWA AI Gateway Layer) and does not involve a language model:- Script and language detection: identifies the input language. Arabic written in Latin letters and digits (Arabizi, for example
shlonkm ya shabab) is transliterated to Arabic script first, so it is classified on meaning rather than spelling. - Marker matching: the text is scored against weighted vocabulary sets for each dialect. Only dialect-specific words count. Words shared across every dialect, such as متى or مرحباً, are deliberately excluded, because they would otherwise pull ordinary Gulf and Egyptian comments toward MSA.
- Confidence scoring: a score from 0 to 1 derived from how many markers matched and how strongly they distinguish one dialect from the others.
The dialect_confidence field
The dialect_confidence score ranges from 0 to 1: