Although mathematically elegant, back-propagation isn't perfect. Instead consider using particle swarm optimization (PSO) to train your neural network; here's how.

To solve this problem, A linear integer programming model is established, the lower bound is constructed, and the particle swarm optimization algorithm is solved. The performance of the proposed ...

Qi Xiong, Hongbo Zhang, Qiusheng Rong, Path Planning Based on Improved Particle Swarm Optimization for AUVs, Journal of Coastal Research, SPECIAL ISSUE NO. 111.

Particle swarm optimization isn't usually seen as the first-choice technique for training a neural network but, as James McCaffrey demonstrates, it's a useful alternative.