Tag Recommendation for Short Arabic Text by Using Latent Semantic Analysis of Wikipedia

Abstract

Social media sites enable users to share items, such as texts and images, and annotate them with freely chosen keywords called tags. However, freedom comes at a cost: uncontrolled vocabulary can result in tag redundancy, ambiguity, sparsity, miss-spilling, and idiosyncrasy, thus impeding more effective organization/retrieval of resources in tagging systems. This work proposes an Arabic Language tag recommender system that exploits the Arabic Wikipedia as background knowledge. Latent semantic analysis was employed to discover hidden semantics between the short text and Wikipedia articles. Apache Spark was used to handle the massive content of Wikipedia and the complex computations of latent semantic analysis which is used to analyze Wikipedia articles into three matrices. Given an Arabic short text as input, the system compares it to the body of the articles and scores them according to their relevance to the short text. Candidate tags are determined from top-scored articles by exploiting articles' titles and categories. The proposed system was assessed over a dataset of 100 tweets covering three different domains. Generated tags were rated by two human experts in each domain. Our system achieved 84.39% mean average precision and 96.53% mean reciprocal rank, revealing the system adequacy and accuracy for tagging Arabic short texts while still has difficulties regarding Arabic language, and affected by frequencies of rare terms. A thorough analysis and discussion of the evaluation results are also presented to address the limitations and strengths as well as the recommendations for future improvements. Keywords: Short text, tag recommender, Arabic Language, Wikipedia, Latent Semantic Analysis, Spark