A FPGA Accelerator for Real-Time 3D Non-rigid Registration Using Tree Reweighted Message Passing and Dynamic Markov Random Field Generation

Non-rigid 3D registration is a technique for matching 3D scans of a scene involving deformable objects. Augmented reality, gesture recognition, medical imaging, and many other computer vision and graphics applications require real-time registration to model deformable or articulated objects. Unfortunately, non-rigid registration is a computationally intensive problem that requires careful optimization to maximize throughput and latency. We present a FPGA+CPU accelerator for real-time non-rigid 3D registration based on Tree Reweighted Message Passing (TRW-S). We overcome memory bound issues and scheduling limitations of conventional TRW-S by dynamically generating the Markov Random Fields. This, along with a bevy of other architectural optimizations, allows us to almost saturate 1024 multipliers in a Arria 10 at 100MHz. We achieve a 600x speed up over baseline TRW-S and our registration architecture has up to 81x energy reduction over a software implementation of our algorithm. We demonstrate the performance of our system by performing real-time (20 scan per second) registration on a complicated surgical scene.