جستجوی مقالات مرتبط با کلیدواژه « word embedding » در نشریات گروه « برق »

In today's information age, efficient document ranking plays a crucial role in information retrieval systems. This article proposes a new approach to document ranking using embedding models, with a focus on the BERT language model to improve ranking results. The proposed approach uses vocabulary embedding methods to represent the semantic representations of user queries and document content. By converting textual data into semantic vectors, the relationships and similarities between queries and documents are evaluated under the proposed ranking relationships with lower cost. The proposed ranking relationships consider various factors to improve accuracy, including vocabulary embedding vectors, keyword location, and the impact of valuable words on ranking based on semantic vectors. Comparative experiments and analyses were conducted to evaluate the effectiveness of the proposed relationships. The empirical results demonstrate the effectiveness of the proposed approach in achieving higher accuracy compared to common ranking methods. These results indicate that the use of embedding models and their combination in proposed ranking relationships significantly improves ranking accuracy up to 0.87 in the best case. This study helps improve document ranking and demonstrates the potential of the BERT embedding model in improving ranking performance.

Using the context and order of words in sentence can lead to its better understanding and comprehension. Pre-trained language models have recently achieved great success in natural language processing. Among these models, The BERT algorithm has been increasingly popular. This problem has not been investigated in Persian language and considered as a challenge in Persian web domain. In this article, the embedding of Persian words forming a sentence was investigated using the BERT algorithm. In the proposed approach, a model was trained based on the Persian web dataset, and the final model was produced with two stages of fine-tuning the model with different architectures. Finally, the features of the model were extracted and evaluated in document ranking. The results obtained from this model are improved compared to results obtained from other investigated models in terms of accuracy compared to the multilingual BERT model by at least one percent. Also, applying the fine-tuning process with our proposed structure on other existing models has resulted in the improvement of the model and embedding accuracy after each fine-tuning process. This process will improve result in around 5% accuracy of the Persian web ranking.

Nowadays, according to the growing volume of electronic documents, the classification of text has attracted the attention of information retrieval researchers. Considering the importance of text classification and the efforts done in this field in several languages in the world, the necessity of Persian text classification is understood. In general, we can classify text classification methods into two classes, including traditional methods (based on feature selection and machine learning) and methods based on deep learning. Deep learning methods, due to the ability of weight sharing, significantly reduce the number of trainable parameters and thus increase generalization and provide better results than other methods. There are a few methods based on deep learning for Persian text classification. In this study, we propose to use CNN and BLSTM with an attention layer for Persian text classification named ParsCNN and ParsBiLSTM. The experimental results on the Hamshahri dataset show that the ParsCNN method has a precision of 0.69, a recall of 0.7, and, an F-score of 0.69; Also, the ParsBiLSTM method has a precision of 0.72, a recall of 0.73 and, an F-score of 0.72, which indicates the methods based on deep Neural Networks have better performance than other approaches.

Today, due to the large volume of opinions published by people in cyberspace, sentiment analysis plays a key role in extracting information. One of the new techniques based on studies has been done to determine the exact polarity of the sentence in sentiment analysis is deep learning algorithms. In this research, two deep learning algorithms, namely RNN and LSTM, has been used to determine sentence polarity in order to achieve more accurate results. Moreover, in the proposed technique, pre-trained word embedding algorithm, namely Wordtovec, was used to determine the semantic relationships between words to increase the accuracy of the proposed method. The proposed method was evaluated on two data sets; airline-tweet and IMDB. The evaluation results show that on the airline-tweet dataset, the proposed method has an accuracy of 0.78 and accuracy of 0.84 on the IMDB data set.

For data mining studies, due to the complexity of doing feature selection process in tasks by hand, we need to send some of labeling to the workers with crowdsourcing activities. The process of outsourcing data mining tasks to users is often handled by software systems without enough knowledge of the age or geography of the users' residence. We use convolutional neural network, for doing classification in six classes: USAGE, TOPIC, COMPARE, MODEL-FEATURE, RESULT and PART-WHOLE. This article extracts the data from the abstract of 450 scientific articles and it is a total of 835 relations. One hundred of these abstracts have been selected by the crowdsourcing. Classification results in this article have been done with a slight improvement in accuracy. In this study, we computed the classification results on a combination of vocabulary vectors with using of 450 abstract relation data (100 crowd source datasets with 350 standards). The results of the implementation of the classification algorithm give us performance improvement. This paper uses the population power to perform preparing data mining works. The proposed method by adding crowdsource data to the previous data was able to obtain better results rather than the top 5 methods.

One of the main processes in most natural language processing (NLP), is named entity recognition (NER). In this regard, some machine learning techniques have been presented that traditionally use manual features. Also, in recent years, deep neural network-based models have been proposed that achieve higher accuracy without relying on huge computations for feature engineering. Thus, in this article, we employ a combination of two deep learning models to capture the properties of the input sentence, including: long short term memory (LSTM) and convolutional neural network (CNN). In this architecture, extracting local features along with global features, more information is acquired for more accurate classification. We evaluate the performance of this architecture on two datasets CoNLL2003 and ACE05; and demonstrate that by adding a word level CNN, useful local properties are extracted that enhance the accuracy of the performance. Finally, we compare the performance of our system with competitors and our superiority is reported.

Process mining provides a bridge between process modeling and analysis on the one hand and data mining on the other hand. Process mining aims at discovering, monitoring, and improving real processes by extracting knowledge from event logs. However, as most business processes change over time (e.g. the effects of new legislation, seasonal effects and etc.), traditional process mining techniques cannot capture such “second-order dynamics” and analyze these processes as if they are in steady-state. Such changes can significantly impact the performance of processes. Hence, for the process management, it is crucial that changes in processes be discovered and analyzed. Process change detection is also known as business process drift detection.All the existing methods for process drift detection are dependent on the size of windows used for detecting changes. Identifying convenient features that characterize the relations between traces or events is another challenge in most methods. In this thesis, we propose an automated and window-independent approach for detecting sudden business process drifts by introducing the notion of trace embedding. Using trace embedding makes it possible to automatically extract all features from the relations between traces. We show that the proposed approach outperforms all the existing methods in respect of its significantly higher accuracy and lower detection delay.

Term mismatch is the most important challege in web information retrieval. The term mismatch problem is defined as differences between user queries and contents of documents while referring to the same topic. Query expansion methods deal with term mismatch by reformulating the queries to increase their term-overlap with relevant documents. In this paper, we proposed a query expansion framework based on a deep Siamese LSTM neural network. In addition, we defined the relevant relatedness for the first time and used this concept to label pairs made from user query and candidate query. Weakly-supervised labeled pairs are utilized in training of the deep Siamese network. The trained Siamese network provides labels for testset pairs in addition to contrastive loss values. The contrastive loss value reflects the cost of pulling together similar pairs. Pairs with minimum contrastive loss values are selected and merged together to form one expanded query. Results of our tests showed that the proposed framework outperforms similar word embedding based query expansion methods.

Named entities recognition is a fundamental task in the field of natural language processing. It is also known as a subset of information extraction. The process of recognizing named entities aims at finding proper nouns in the text and classifying them into predetermined classes such as names of people, organizations, and places. In this paper, we propose a named entity recognizer which benefits from neural network-based approaches for both word representation and entity tagging. In the word representation part of the proposed model, two different vector representations are used and compared: (1) the semantic representation of words based on their context using word2vec continues skip-gram model, and (2) the semantic representation of words based on their context as well as characters forming them using fasttext. While the former model captures the semantic concepts of words, the latter one considers the morphological similarity of words as well. For the entity identification, a deep Bidirectional Long Short Term Memory (BiLSTM) network is used. Using LSTM model helps to consider the history of text when predicting entities, while the BiLSTM model expands this idea by benefiting from the history from both sides of the context. Moreover, inline of the present research, an annotated corpus containing 3000 abstracts (90000 tokens) from the Persian Wikipedia is provided. In contrast to the available datasets in the field, which includes up to 7 label types, the new dataset contains 15 different labels, namely person individual, person group, organizations, locations, religions, books, magazines, movies, languages, nationalities, events, jobs, dates, fields, and other. Developing this dataset will be an important step in promoting future research in this field, especially for the tasks such as question answering that need wider range of entity types. The results of the proposed system show that by using the introduced model and the provided data, the system can achieve 72.92 F-measure.