Proposing Novel Activation Functions for Complex-Valued Neural Networks and their Applications on Real-Valued Classification Problems

One of the primary challenges in complex-valued neural networks (CVNNs) is to find an appropriate activation function (AF) that should be differentiable and bounded. The AF should properly map the complex domain into real-valued outputs, when the CVNN is used to process real-valued problems. In this paper, we proposed two novel AFs that well satisfied the above conditions. The proposed AFs saturate in four regions and unlike the simple two-layered perceptron, they are able to solve linear non-separable problems. Weight adjustment formulas are developed, and the learning and testing processes are described for both networks. To evaluate the performance of the proposed scheme, two readily available labeled data sets on diabetes and breast cancer are used to detect the respective illnesses. It has been shown that the proposed CVNNs have simpler structures and faster convergence rate than an standard multi-layered real-valued perceptron. The proposed scheme achieves correct diagnosis rates of 80% and 95% for diabetes and breast cancer, respectively in the corresponding data sets.