MentalMARBERT: Domain-Adaptive Pre-training and Two-Stage Fine-Tuning for Arabic Mental Health Disorders Detection

The study introduces MentalMARBERT, a framework for detecting mental health disorders in Arabic social media text, which involves Domain-Adaptive and Task-Adaptive Pretraining (DAPT and TAPT) of three Arabic models: AraBERT, CAMeLBERT, and MARBERT. A new dataset of 50,670 annotated tweets was created, and the best-performing configuration utilized a hierarchical two-stage architecture with full fine-tuning, achieving a macro-F1 score of 0.861 and an accuracy of 0.877. This work highlights the importance of domain-specific adaptations and advanced classification techniques in improving NLP tasks for underrepresented languages like Arabic.