Convolutional neural networks with fractional order gradient method

This paper proposes a fractional order gradient method for the backward propagation of convolutional neural networks. To overcome the problem that fractional order gradient method cannot converge to real extreme point, a simplified fractional order gradient method is designed based on Caputo's definition. The parameters within layers are updated by the designed gradient method, but the propagations between layers still use integer order gradients, and thus the complicated derivatives of composite functions are avoided and the chain rule will be kept. By connecting every layers in series and adding loss functions, the proposed convolutional neural networks can be trained smoothly according to various tasks. Some practical experiments are carried out in order to demonstrate the effectiveness of neural networks at last.

Optimal Randomness in Swarm-based Search

Swarm-based search has been a hot topic for a long time. Among all the proposed algorithms, Cuckoo search (CS) has been proved to be an efficient approach for global optimum searching due to the combination of L\'{e}vy flights, local search capabilities and guaranteed global convergence. CS uses L\'{e}vy flights which are generated from the L\'{e}vy distribution, a heavy-tailed probability distribution, in global random walk to explore the search space. In this case, large steps are more likely to be generated, which plays an important role in enhancing the search capability. Although movements of many foragers and wandering animals have been shown to follow a L\'{e}vy distribution, investigation into the impact of different heavy-tailed probability distributions on CS is still insufficient up to now. In this paper, four different types of commonly used heavy-tailed distributions, including Mittag-Leffler distribution, Pareto distribution Cauchy distribution, and Weibull distribution, are considered to enhance the searching ability of CS. Then four novel CS algorithms are proposed and experiments are carried out on 20 benchmark functions to compare their searching performance. Finally, the proposed methods are used to system identification to demonstrate the effectiveness.