This thesis concerns the application of artificial neural networks to solve optimization and dynamical control problems. A general framework for artificial neural networks models is introduced first. Then the main feedforward and feedback models are presented. The IAC (Interactive Activation and Competition) feedback network is analysed in detail. It is shown that the IAC network, like the Hopfield network, can be used to solve quadratic optimization problems. A method that speeds up the training of feedforward artificial neural networks by constraining the location of the decision surfaces defined by the weights arriving at the hidden units is developed. The problem of training artificial neural networks to be fault tolerant to loss of hidden units is mathematically analysed. It is shown that by considering the network fault tolerance the above problem is regularized, that is the number of local minima is reduced. It is also shown that in some cases there is a unique set of weights that minimizes a cost function. The BPS algorithm, a network training algorithm that switches the hidden units on and off, is developed and it is shown that its use results in fault tolerant neural networks. A novel non-standard artificial neural network model is then proposed to solve the extremum control problem for static systems that have an asymmetric performance index. An algorithm to train such a network is developed and it is shown that the proposed network structure can also be applied to the multi-input case. A control structure that integrates feedback control and a feedforward artificial neural network to perform nonlinear control is proposed. It is shown that such a structure performs closed-loop identification of the inverse dynamical system. The technique of adapting the gains of the feedback controller during training is then introduced. Finally it is shown that the BPS algorithm can also be used in this case to increase the fault tolerance of the neural controller in relation to loss of hidden units. Computer simulations are used throughout to illustrate the results.