Design of a filter bank-based convolutional neural network for handwritten digit images classification

In recent years, convolutional neural networks (CNNs) have been increasingly used in various applications of machine vision. CNNs simulate the function of the brain's visual cortex and have a powerful structure for analyzing visual images. However, the diversity of digital images, their content, and their features necessitate that CNN networks are specially designed, and their parameters are carefully adjusted to achieve higher efficiency in any classification problem. In this regard, in many previous studies, researchers have attempted to increase the efficiency of the CNNs by setting their adjustable parameters more accurately.

New method In this study, we presented a novel initializing method for the kernels of the first convolutional layer of the CNN networks. We designed a filter bank with specialized kernels and used them in the first convolution layer of the proposed models. These kernels, compared to the random kernels in traditional CNNs, extract more effective features from the input images without increasing the computational cost of the network, and improve the classification accuracy by covering all the important characteristics.

The dataset used in this paper was the MNIST database of handwritten digits. We examined the performance of CNN networks when three different types of kernels were used in their first convolution layer. The first group of kernels had constant coefficients; the second group had random coefficients, and finally, the kernels of the third group were specially designed to extract a wide range of image features. Our experiments on a single-layer CNN network with three types of kernels (constant numbers, random numbers, and filter-bank) showed the average classification accuracy of MNIST images in 50 times of network training to be 74.94%, 86.47%, and 91.89%, respectively, and for a three-layer CNN network, 88.82%, 96.16%, and 99.14%, respectively. Comparison with existing methods Compared to the kernels with randomized coefficients, the use of specialized kernels in the first convolution layer of the CNN networks has several important advantages: 1) They can be designed to extract all important features of the input images, 2) They can be designed more effectively based on the problem in hand, 3) They cause the training to start from a more appropriate point, and in this way, the speed of training and the classification accuracy of the network increase.

This study provides a novel method for initializing kernels in convolution layers of CNN networks to enhance their performance in image classification works. Our results show that compared to random kernels, the kernels used in the proposed models extract more effective features from the images at different frequencies and increase the classification accuracy by starting the training algorithm from a more appropriate point, without increasing the computational cost. Therefore, it can be concluded that the initial coefficients of the convolution layer kernels are effective on the classification accuracy of CNN networks, and by using more effective kernels in the convolution layers, these networks can be made specific to the problem and, in this way, increase the efficiency of the network.