Fish Shoals Behavior Detection Based on Convolutional Neural Network and Spatiotemporal Information

Behavior is the first visible change in an animal species after exposure to its own or environmental stressors and is a sensitive indicator. Fish are social animals, and the abnormality of group behavior is more indicative about a particular event than individual behavior, providing more effective informeqation about environmental or group social changes. The group behavior is not only reflected in the spatial distribution, but also reflected in the temporal behavior of the group and individual movement changes under the influence of pressure factors. This paper proposes a group behavior discrimination method based on convolutional neural network and spatiotemporal information fusion, which intends to make use of the prominent performance of convolutional neural network in image recognition and state classification, and imitating the attentional mechanism of ventral channel and dorsal channel when the human brain processes visual signals. Some pressure environments are made in laboratory, the behavior states of fish shoals are recorded, and the sample database of shoals’ behavior state is established by combining the spatial information of shoals’ spatial distribution with the time information reflected in the movement behavior. A simple convolutional neural network is constructed to quickly identify the behavior state of fish shoals. The effects of bath size and training epoch on network training speed and recognition accuracy are discussed, and the visualization of the intermediate data of the convolutional neural network is studied. Shown from the results of experiments of this paper, different behavior states of fish shoals can be recognized and classified effectively by using the simple convolutional neural network and spatiotemporal fusion images. What’s more, from the visualization of network intermediate data, it is found that the convolutional neural network has a higher discrimination power to the image edge feature than the image gray-value feature.