Motion prediction via online instantaneous frequency estimation for vision-based beating heart tracking

An efficient prediction scheme based on the dual Kalman filter is developed.A new orthogonal fusion algorithm is proposed to measure heart motion frequencies.Directional difference of breathing and heartbeat motion is taken into account.In vivo dataset measured from real beating heart surgeries is used for validation. The beating heart tracking based on stereo endoscope remains challenging due to highly dynamic scenes and poor imaging conditions in minimally invasive surgery. This paper proposes a new prediction method for robust tracking of heart motion. The dual time-varying Fourier series is used for modeling the motion of points of interest (POI) on heart surfaces, which is driven jointly by breathing and heartbeat motion. A dual Kalman filtering scheme is used to estimate the frequencies and Fourier coefficients of the model respectively. A novel orthogonal decomposition algorithm is developed to measure the instantaneous frequencies of breathing and heartbeat motion online from the 3D trajectory of the POI. The difference in direction between breathing and heartbeat motion is exploited by using principal component analysis on the past trajectory, and optimal 1D principal component signals are extracted for measuring the corresponding frequencies. The frequencies calculated from the orthogonal subbands are fused based on an additive noise model for optimal frequency measurement. The proposed method is evaluated and compared with other available prediction methods based on the simulated data and the real-measured signals from the videos recorded by the daVinci surgical robot. The prediction algorithm is finally incorporated into a well-established visual tracking method to handle long-term occlusions.