Digital Object Identifier: 10.1109/ TNNLS.2013.2293637 “There has been a recent increase in researchers’ attention on connectionist models composed by several hidden layers. In fact, experimental results and heuristic considerations suggest that deep architectures are more suitable than shallow ones for modern applications, facing very complex problems, e.g., vision and human language understanding. However, the actual theoretical results supporting such a claim are still few and incomplete. In this paper, a new approach is proposed to study how the depth of feedforward neural networks impacts on their ability in implementing high complexity functions. First, a new measure based on topological concepts is introduced, aimed at evaluating the complexity of the function implemented by a neural network, used for classification purposes. Then, deep and shallow neural architectures with common sigmoidal activation functions are compared, by deriving upper and lower bounds on their complexity, and studying how the complexity depends on the number of hidden units and the used activation function. The obtained results seem to support the idea that deep networks actually implement functions of higher complexity, so that they are able, with the same number of resources, to address more difficult problems.”
[1]
John T. Rickard,et al.
Type-2 Fuzzy Sets as Functions on Spaces
,
2010,
IEEE Transactions on Fuzzy Systems.
[2]
Lotfi A. Zadeh,et al.
Fuzzy logic = computing with words
,
1996,
IEEE Trans. Fuzzy Syst..
[3]
Hani Hagras,et al.
Toward General Type-2 Fuzzy Logic Systems Based on zSlices
,
2010,
IEEE Transactions on Fuzzy Systems.
[4]
Ioannis K. Vlachos,et al.
Subsethood, entropy, and cardinality for interval-valued fuzzy sets - An algebraic derivation
,
2007,
Fuzzy Sets Syst..
[5]
Jerry M. Mendel,et al.
Encoding Words Into Interval Type-2 Fuzzy Sets Using an Interval Approach
,
2008,
IEEE Transactions on Fuzzy Systems.
[6]
Jerry M. Mendel,et al.
Interval Type-2 Fuzzy Logic Systems Made Simple
,
2006,
IEEE Transactions on Fuzzy Systems.