Implementation of a Real-Time Image-Based Vibration Detection and Adaptive Filtering on an FPGA

In this paper, we propose and implement a field-programmable gate array (FPGA) system which extracts a vibration component of a desired frequency band from moving images in real-time, aiming at application to image-based vibration suppression such as microsurgery assistance systems. The technical challenges to this end are two-fold: fast and robust detection of vibration components in given moving images and zero-phase band-pass filtering for a desired frequency band. For the former, we employ a statistical approach using dense optical flow to derive frequency components, and design a custom optical flow computing hardware with the Lucas-Kanade (LK) method. For the latter, we implement a sort of adaptive band-pass filters called a bandlimited multiple Fourier linear combiner (BMFLC), which can recompose input signals as a mixture of sinusoidal signals with multiple frequencies in a band with no phase delay. Both designs are implemented in a deeply pipelined manner on a Xilinx Kintex-7 XC325T FPGA, without using any external memories. Empirical experiments reveal that the proposed system extracts a vibration component of high-frequency tremors in hand motions, while intentional low-frequency motions are successfully filtered out. The system processes VGA moving images at 60 FPS, with a delay of less than 1 us for the BMFLC, suggesting effectiveness of the deep pipelined architecture.