Towards robust specularity detection and inpainting in cardiac images

Computer-assisted cardiac surgeries had major advances throughout the years and are gaining more popularity over conventional cardiac procedures as they offer many benefits to both patients and surgeons. One obvious advantage is that they enable surgeons to perform delicate tasks on the heart while it is still beating, avoiding the risks associated with cardiac arrest. Consequently, the surgical system needs to accurately compensate the physiological motion of the heart which is a very challenging task in medical robotics since there exist different sources of disturbances. One of which is the bright light reflections, known as specular highlights, that appear on the glossy surface of the heart and partially occlude the field of view. This work is focused on developing a robust approach that accurately detects and removes those highlights to reduce their disturbance to the surgeon and the motion compensation algorithm. As a first step, we exploit both color attributes and Fuzzy edge detector to identify specular regions in each acquired image frame. These two techniques together work as restricted thresholding and are able to accurately identify specular regions. Then, in order to eliminate the specularity artifact and give the surgeon a better perception of the heart, the second part of our solution is dedicated to correct the detected regions using inpainting to propagate and smooth the results. Our experimental results, which we carry out in realistic datasets, reveal how efficient and precise the proposed solution is, as well as demonstrate its robustness and real-time performance.