DEVIATION THEOREMS FOR SOLUTIONS OF LINEAR ORDINARY DIFFERENTIAL EQUATIONS AND APPLICATIONS TO PARALLEL COMPLEXITY OF SIGMOIDS

By a sigmoid with a depth d we mean a computational circuit with d layers in which rational operations are admitted at each layer, and to jump to the next layer the substitution of a function computed at the previous layer into an arbitrary real solution of a linear ordinary differential equation with the polynomial coefficients is admitted. Sigmoids appear as a computational model for neural networks. The deviation theorem is proved which states that for a (real) function 0 6≡ f computed by a sigmoid with a depth (or parallel complexity) d there exist c > 0 and an integer n such that the inequalities (exp(· · · (exp(c|x|n) · · · ) ≤ |f(x)| ≤ exp(· · · (exp(c|x|n) · · · )))) hold everywhere on the real line except for a finite measure set, where the iteration of the exponential function is taken d times. One can treat the deviation theorem as an analogue of Liouvillean theorem (on the bound of the difference of algebraic numbers) for solutions of ordinary differential equations. Also we estimate the numbers of zeroes of f in the intervals.