جستجوی مقالات مرتبط با کلیدواژه "persian news dataset" در نشریات گروه "برق"

Due to the ever-increasing amount of data, the amount of textual data is also growing at a high speed. Extracting information from these textual data is one of the necessities of today's information-based world. Text classification is one of the methods of obtaining information from this massive data. In this research, using a standard dataset of Persian news, which included five features in more than 86 thousand news, we investigated the performance of the logistic regression algorithm in the classification of Persian text and also compared it with other similar works. Considering the steps of creating a text category, we have explained the method used in the vectorization section and also stated the importance of the pre-processing section, especially the method used in tagging and converting sub-tags to main ones. In the final evaluation, by changing the algorithm's parameters and modifying the news tags, we reached the desired result of 95% in the accuracy criterion for the text classification of the Persian news dataset.

Although the lack of data is one of the important challenges for research in the field of natural language processing, but this challenge is more acute in the Persian language, so finding a high-quality and comprehensive dataset in the Persian language is a difficult task. In addition to that, having some problems such as the ability to categorize and not complying with the storage standard are among the problems of the existing datasets, each of which can affect the learning rate of the model, the results, and the error rate in the experiments. For this reason, all these reasons made us seek to collect and prepare a dataset that covers all such problems and reduces the amount of error when using data in different models. In this research, we have designed and used a crawler to collect textual data. By crawling on one of the news bases, it has been able to collect data sets in five columns: title, summary, text, tag, and publication date. The textual data has been normalized with the help of one of the Persian language libraries in the Python programming language and stored in csv and xml formats and made available to fellow researchers. The tags in this dataset include 13 main tags of sports, art and media, culture, science and progress, political, foreign policy, life, family, society, education and training, international, economic and provinces. Among the tasks that can be done on this data set are text classification, text extraction, text summarization and title recognition. Also, one of the prominent features of this data set is its comprehensiveness, the amount of suitable data, the existence of useful features, having unique features, as well as storage in a standard format. This dataset is a product of the Language Processing Department of Imam Hossein Comprehensive University and can be downloaded and used through the link mentioned in the footnote of the next page and with respect to copyright.