Optimization in Neural Networks

The idea of Artificial Neural Networks (ANN) is to build a massively connected and highly parallel system from simple processing units. This idea stems from real biological systems. These units are regarded as simplification of biological neurons what explains the ANN term. The information in ANN is stored in “weights” of connections between the units. It is assumed that ANN “adapts” to a problem by changing these weights.