Low power feature-extraction smart CMOS image sensor design

Digital image sensors have been utilized in various applications including, but not limited to, consumer electronics, machine vision, and security surveillance. In the early days of digital cameras, charge-coupled-device (CCD) technology was dominant because of its superior imaging quality. In recent years, with the advancement of complementary-metal-oxide-semiconductor (CMOS) technology, CMOS image sensors have prevailed due to their lower power consumption and inexpensive fabrication cost. The design of commercial image sensors focuses on the improvement of imaging quality by increasing spatial resolution with smaller pixel size. When sensors without an on-chip computation capability are applied to the computer vision field, they produce massive amounts of raw image data. To extract valuable image features, such redundant raw data has to be transmitted and then processed by the vision processing system. This process wastes tremendous hardware resources and causes significant power consumption. Therefore, there is a growing demand for the smart image sensor that can perform on-chip image processing to directly export valuable image features such as spatial contrast or temporal motion. In many computer vision applications, motion features are widely used to identify active objects from stationary backgrounds. Once active regions with motion activities are extracted, more advanced processing such as image compression, object segmentation, and pattern recognition can be implemented on the acquired images. Visual motion detection can be achieved by measuring light intensity changes along with xxi time on every pixel. Light intensities can be transformed into electrical signals in the form of currents, charges, or voltages. By detecting temporal variations in these signals with dedicated circuits, a motion-detection function can be implemented on image sensors. Various smart image sensors with on-chip motion feature-extraction capabilities have been reported in recent years. They can be classified into two main categories: “discrete mode” and “continuous mode”. In the first mode, an analog intensity image is acquired by integrating incident light within a certain period. A comparison of two consecutive frames can realize the motion-detection function. However, this algorithm suffers from a severe aliasing effect due to the discrete differentiation between two separate frames. In the “continuous mode”, incident light is directly converted into photo currents or intensity voltages without the integrating operation. By applying continuous differentiation on these signals, motion features can be extracted more efficiently. In order to report motion activities in real time, researchers innovatively developed an asynchronous address-event-representation (AER) strategy to export motion address events from smart image sensors to post processors. Unfortunately, image processing on address events is different with the conventional frame-based strategy because of their asynchronous nature, and specific event-based algorithms have to be customized based on applications. The aim of this work is to design a smart CMOS image sensor that can extract and process motion features on the same chip so as to simplify the entire visual system with minimal computation burden. In order to achieve this ultimate goal, several smart motion-detection image sensors have been developed. The first image sensor was designed based on the temporal difference algorithm to on-chip export motion features, and this sensor is a compact visual system that integrates motion detection and feature processing on the same chip. In order to maximize photon sensitivity in every pixel, a feature-extraction image sensor based on an emerging 3D integration xxii technology was developed. The sensor can extract either temporal motion or spatial contour in real time. Also, an event-clustering image sensor specialized for object tracking applications was proposed. This sensor continuously detects motion events and on-chip processes them in parallel to export specific “event-flow” features. Simulation results show that this image sensor significantly simplifies and accelerates visual object tracking systems.