Persian language challenges in the digital environment and their impact on the effectiveness of automated text processing and information retrieval

Research Due to its special characteristics and lacking a standard writing style, Persian language faces with difficulties in coping with electronic media, seemingly affecting the effectiveness of the information retrieval. Reviewing previous literature on the subject, the present study attempts to study the challenges, their impact on the effective information retrieval and the solutions proposed to overcome them.Research Significance: A comprehensive review on the Persian Language challenges in digital environment may highlight the knowledge gathered so far on the subject and uncover its deficiencies and progress.Research The present study applies a documentary method with a text analysis approach, which is a qualitative method. The reviewed studies report a large number of inconsistencies and variations in Persian writing, seeming to seriously affect the effectiveness of information retrieval regarding precision, false drop or recall. It is, therefore, required that Persian information retrieval systems normalize the morphological and grammatical inconsistencies and variations. The studies offer some solutions to eliminate or lessen the negative effects, namely standardizing Persian writing style, using pre-defined lists, equipping the databases with thesauri. Although failing to encompass all the challenges, they seem to be more or less effectual. Specially, preparing Persian writing style standards, spelling dictionaries and thesauri, search manuals or guides, and designing subject-specific information retrieval algorithms can be useful. The human-based solutions require serious training of and contribution from text writers (including typists and users) and are a long-term and expensive process. Consequently, it is vital to move toward automatic text processing and indexing techniques.