Automated rapid thermal imaging systems technology

A major source of energy savings occurs on the thermal envelop of buildings, which amounts to approximately 10% of annual energy usage in the United States. To pursue these savings, energy auditors use closed loop energy auditing processes that include infrared thermography inspection as an important tool to assess deficiencies and identify hot thermal gradients. This process is prohibitively expensive and time consuming. I propose fundamentally changing this approach by designing, developing, and deploying an Automated Rapid Thermal Imaging Systems Technology (ARTIST) which is capable of street level drive-by scanning in real-time. I am doing for thermal imaging what Google Earth did for visual imaging. I am mapping the world's temperature, window by window, house by house, street by street, city by city, and country by country. In doing so, I will be able to provide detailed information on where and how we are wasting energy, providing the information needed for sound economic and environmental energy policies and identifying what corrective measures can and should be taken. The fundamental contributions of this thesis relates to the ARTIST. This thesis will focus on the following topics: * Multi-camera synthetic aperture imaging system * 3D Radiometry * Non-radiometric infrared camera calibration techniques * Image enhancement algorithms o Hyper Resolution o Kinetic Super Resolution o Thermal Signature Identification o Low-Light Signal-to-Noise Enhancement using KSR Thesis Supervisor: Alexander H. Slocum Tile: Professor of Mechanical Engineering Automated Rapid Thermal Imaging Systems Technology Page 3 Automated Rapid Thermal Imaging Systems Technology Page 4 Foreword & Acknowledgement As I sit here on two days away from my thesis defense, I am reminded of what I am doing here and my reasons for being at MIT. It has definitely been a long journey for me, dating back to 1994 when I first arrived at MIT as a freshman. It is indeed, long ago, almost 18 years to be exact, just about the age of an incoming freshman next year. It does make me put things in perspective and I would like to share some of my stories, experiences, and begin to thank the many people who made this journey possible for me. First and foremost, I would like to thank my parents, who, through their sacrifices and efforts have made it possible for me exist and have the opportunities in front of me today. My parents, when I was 2.5 years old, decided to make the ultimate sacrifice at the time, leaving their home country and place of birth, Vietnam, to take me and my uncle (my mother's younger brother) to leave Vietnam as boat refugees. It was a perilous time in the late 1970's and they had to leave everything behind; their home, their valuables and belongings, only to carry what they could on their backs literally. I will save the details for a later time but would summarize their actions brave, courageous, and risky. For them to have left the way they did and I'm not sure if I would have the determination to do the same. I am grateful and thankful to my mother and father for finding the will and way to survive and get everyone safely to our destination, the United States. We arrived here as immigrants in December of 1979. Both my parents were relatively common people neither of them with any significant amount of schooling, my mother left high school so work at a textile factory so she can take care of her family and my father dropping out of grade school. This made life difficult for them the first few years in a new land, working relatively laborious jobs as a janitor and doing whatever it took to survive and adjust to life in a new land. There are countless examples of their sacrifices throughout the first 18 years of life; my father delivering my daily newspaper route so I can earn money to take computer classes and my mother working late into the night as a stay-at-home sewer (making aloha shirts and shorts). I will leave the stories and details for a later time. The reason why I share this is to make it clear who I am dedicating my degrees too. I wish I dedicate all my success and 3 MIT degrees to my parents, who, through me, have achieved the peaks of education and who through me, will soon possess my third MIT degree, a PhD in Mechanical Engineering. Automated Rapid Thermal Imaging Systems Technology Page 5 Next, I will thank individual people(s) who have made a difference in my life in chronological order. My apologies for not prioritizing this list as it is fairly long and I want to make sure I am complete in acknowledging those who have made a lasting difference in my life. To begin, I would like to acknowledge my siblings, Napoleon and Elizebeth Phan, who are both some of most loyal and loving siblings an older brother can have. To this date, my siblings still acknowledges me in the old fashioned way in Vietnamese traditions, calling me "Anh hai", which literally means brother number two or eldest brother (in Vietnamese traditions, we start with the number two to mean the oldest). They have been good to me over the years. I am proud of the both of them; my sister for her hard work, considerate and caring nature and to my brother, for his courage to face challenges and new experiences. Both in my eyes, are some of the best siblings one can ask for. In Elementary school, one teacher stands out, or, to be more specific, one LibrarianMs Muronaga from Lincoln Elementary. Ms Muronaga who pushes to encourage students to be leaders and provides them with the opportunities to shine and be recognized in early in life. Ms Muronaga ran a 'library aide' program where she selects hard working students to aid her in running the library, giving them privileges to leave class early to perform their duties. My particular duty was to take care of the computer lab; to setup and take-down the computers every day at school. This is how I got exposed to working with computers early on that and taking extra computer programming night classes. In middle school, two teachers stood out in my eyes, Ms. Oda and Mr. Larson, both from the Science Department at Stevenson Intermediate School. Both of them gave me the opportunity to lead the school's toothpick bridge building team as captain, designing competitions and challenges (Louvee air car), and then recognizing and awarding me the Outstanding Science student award. I am very grateful of their early recognition of my science skills, which only served to encourage me to study the sciences. I am particularly grateful to Mr. Larson for doing two things; 1) he would hold regular games in the class whenever we had spare time to do multiplication challenges where he would ask, what is XX times YY, and the first person to raise their hands would win a point for their side of the class (luckily, I was always the first and then after a while, he would disallow my participation for lack of fairness to the other students) and 2) he showed me a video of the MIT 2.70 Automated Rapid Thermal Imaging Systems Technology Page 6 challenge in 7 th grade where I saw Thomas Massey and Professor Woodie Flowers from MIT, which became my sole obsession and inspiration for getting into MIT (just so I can do the competition). This video became the driving force for the next 6 years of my life, as I competed in competition after competition, year after year, to prepare myself for MIT. Little did I know how difficult it was to get into MIT at the time. In high school, I had many teachers who supported and natured my education. Three of them stand out in particular; Mr. Lau, Mr. Ching, and Ms. Kadooka. Mr. Lau was my electronics teacher, who came into teaching when I was a juniorhe became an advisor for several solar car competitions that I competed in, spending countless hours with the students, helping us design and build our competition machines. Mr. Ching who pushed me to become a better person, who had compassion for a student who physically sub-par, and who trained me to become a decent volleyball athlete. Mr. Ching would make me do push ups and sit ups during recess to build my strength. Over a period of three years, he trained me from a volleyball nobody into an important member of the team where we won the division championship. Mr. Ching spent countless summers training and teaching us how to think competitively on the court. On top of that, he was also a friend and life mentor who taught us to be good people off the court as well. Finally, Ms. Kadooka, who has always been like a second mother to me and taught me Physics. She was always pushing me to think, understand, and learn science in a more intuitive manner. Over the years, she has been one of my biggest believers and supporter. The most memorable event for me from Ms. Kadooka was her gracious letter to MIT on my behalf, where she praised me for being one of her best studentsI never forgot this as she has had many students over her career and to be considered one of the best, I was honored and grateful. For all the help through my schooling, I am thankful to my teachers. There are many who have been instrumental in my development in elementary, middle, and high-school. If I have forgotten to mention you specifically, I would like to say "thank you". I would like to take this brief moment to mention a very old friend of mine, who when I was 17, called him up for a last minute MIT interview, explaining that I was applying early, the deadline was 2 weeks away, and that I needed an interview to complete the early action application. This interviewer and life long friend, is Mr. Richard Lowe, whom over the years have kept in touch with me, supported, and truly believed in me in so many ways. Richard interviewed me, wrote 5 pages of Page 7 Automated Rapid Thermal Imaging Systems Technology of gracious comments and somehow influenced MIT admissions enough to get me in early. I am grateful for this life changing event. I remember leaving for college shortly after I turned 18arr

[1]  Joel A. Levitt,et al.  The Influence of Tire Damping in Quarter Car Active Suspension Models , 1991 .

[2]  John R. Liebman,et al.  A Guide to Export Controls for the Non-Specialist , 2006 .

[3]  P Boccacci,et al.  Super-resolution in computational imaging. , 2003, Micron.

[4]  Peyman Milanfar,et al.  Fundamental performance limits in image registration , 2004, IEEE Trans. Image Process..

[5]  Raja Nassar,et al.  Empirical Modeling of Air Tightness in Residential Homes in North Louisiana , 2014 .

[6]  Li Zhu,et al.  Detailed Energy Saving Performance Analyses on Thermal Mass Walls Demonstrated in a Zero Energy House , 2009 .

[7]  Jon Hand,et al.  CONTRASTING THE CAPABILITIES OF BUILDING ENERGY PERFORMANCE SIMULATION PROGRAMS , 2008 .

[8]  Michael Elad,et al.  Advances and challenges in super‐resolution , 2004, Int. J. Imaging Syst. Technol..

[9]  Jan-Michael Frahm,et al.  Robust 6DOF Motion Estimation for Non-Overlapping, Multi-Camera Systems , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[10]  Jon Hand,et al.  Improving non-geometric data available to simulation programs , 2008 .

[11]  Ty A. Newell,et al.  Comparison of Blower Door and Tracer Gas Testing Methods for Determination of Air Infiltration Rates Through Building Envelopes at Normal Operating Conditions , 2011 .

[12]  Long N. Phan Collision avoidance via laser rangefinding , 1999 .

[13]  Moncef Krarti,et al.  Energy Audit of Building Systems : An Engineering Approach , 2000 .

[14]  Zhiqiang John Zhai,et al.  Performance of coupled building energy and CFD simulations , 2005 .

[15]  Xiang Zhu,et al.  Automatic Parameter Selection for Denoising Algorithms Using a No-Reference Measure of Image Content , 2010, IEEE Transactions on Image Processing.

[16]  Min Zhou,et al.  Integrated navigation system and experiment of a low-cost and low-accuracy SINS/GPS , 2007 .

[17]  Christian Früh,et al.  Google Street View: Capturing the World at Street Level , 2010, Computer.

[18]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[19]  Fumio Yamazaki,et al.  Fundamental study on the response characteristics of drivers during an earthquake based on driving simulator experiments , 2004 .

[20]  Allen M. Waxman,et al.  Contour Evolution, Neighborhood Deformation, and Global Image Flow: Planar Surfaces in Motion , 1985 .

[21]  P. Jacobus,et al.  Energy Information Administration New Releases, July--August 1990 , 1990 .

[22]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[23]  A. Rogalski Infrared detectors: an overview , 2002 .

[24]  José A. Sobrino,et al.  Angular variation of thermal infrared emissivity for some natural surfaces from experimental measurements. , 1999, Applied optics.

[25]  C. W. Groetsch,et al.  The theory of Tikhonov regularization for Fredholm equations of the first kind , 1984 .

[26]  Abdelaziz Laouadi Development of a radiant heating and cooling model for building energy simulation software , 2004 .

[27]  Gabor T. Herman,et al.  Fundamentals of Computerized Tomography: Image Reconstruction from Projections , 2009, Advances in Pattern Recognition.

[28]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[29]  Feng Li,et al.  A hybrid camera for motion deblurring and depth map super-resolution , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Denis Pomorski,et al.  GPS/IMU data fusion using multisensor Kalman filtering: introduction of contextual aspects , 2006, Inf. Fusion.

[31]  M. P. Stoll,et al.  Angular variation of land surface spectral emissivity in the thermal infrared: laboratory investigations on bare soils , 1991 .

[32]  Ajit Singh,et al.  An estimation-theoretic framework for image-flow computation , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[33]  Michael Elad,et al.  Robust shift and add approach to superresolution , 2003, SPIE Optics + Photonics.

[34]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[35]  Carlo Tomasi,et al.  Comparison of approaches to egomotion computation , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Hans-Hellmut Nagel,et al.  Displacement vectors derived from second-order intensity variations in image sequences , 1983, Comput. Vis. Graph. Image Process..

[37]  A. Verri,et al.  A computational approach to motion perception , 1988, Biological Cybernetics.

[38]  Susumu Hattori,et al.  Establishing a practical temperature standard by using a narrow-band radiation thermometer with a silicon detector , 1983 .

[39]  A. Slocum,et al.  Precision Machine Design , 1992 .

[40]  Michael Elad,et al.  Multiframe demosaicing and super-resolution of color images , 2006, IEEE Transactions on Image Processing.

[41]  William L. Wolfe,et al.  Introduction to infrared system design , 1996 .

[42]  John R. Schott Incorporation Of Angular Emissivity Effects In Long Wave Infrared Image Models , 1986, Optics & Photonics.

[43]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[44]  Peter Saunders,et al.  Calibration and use of low-temperature direct-reading radiation thermometers , 2009 .

[45]  F. E. Nicodemus Reflectance nomenclature and directional reflectance and emissivity. , 1970, Applied optics.

[46]  J. Peter Clinch,et al.  Valuing improvements in comfort from domestic energy-efficiency retrofits using a trade-off simulation model , 2003 .

[47]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[48]  Fumio Yamazaki,et al.  Seismic response analysis on the stability of running vehicles , 2002 .

[49]  Xinjian Yi,et al.  Characterizations of VO2-based uncooled microbolometer linear array , 2001 .

[50]  Göran Stemme,et al.  Performance model for uncooled infrared bolometer arrays and performance predictions of bolometers operating at atmospheric pressure , 2008 .

[51]  Kevin J. Lomas,et al.  Empirical validation of building energy simulation programs , 1997 .

[52]  Omar M. Al-Rabghi,et al.  Energy simulation in buildings: overview and BLAST example , 2001 .

[53]  James N Caron Rapid supersampling of multiframe sequences by use of blind deconvolution. , 2004, Optics letters.

[54]  R. A. Wood Uncooled Microbolometer Infrared Sensor Arrays , 2001 .

[55]  Corina Sandu,et al.  Multibody Dynamics Modeling and System Identification of a Quarter-Car Test Rig with McPherson Strut Suspension , 2007 .

[56]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[57]  George Vosselman,et al.  3 D RECONSTRUCTION OF ROADS AND TREES FOR CITY MODELLING , 2003 .

[58]  Elizabeth Kossecka,et al.  Multi-dimensional heat transfer through complex building envelope assemblies in hourly energy simulation programs , 2002 .

[59]  Stephen Lin,et al.  Correction of Spatially Varying Image and Video Motion Blur Using a Hybrid Camera , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Philip E. Howard,et al.  DRS U6000 640x480 VOx uncooled IR focal plane , 2002, SPIE Defense + Commercial Sensing.

[61]  Jan Flusser,et al.  Resolution enhancement via probabilistic deconvolution of multiple degraded images , 2006, Pattern Recognit. Lett..

[62]  Jordan E. Segall Google Street View: Walking the Line of Privacy-Intrusion upon Seclusion and Publicity Given to Private Facts in the Digital Age , 2010 .

[63]  Antonia Moropoulou,et al.  Emissivity measurements on historic building materials using dual-wavelength infrared thermography , 2001, SPIE Defense + Commercial Sensing.

[64]  V. Székely,et al.  A new evaluation method of thermal transient measurement results , 1997 .

[65]  Lawrence A. Bergman,et al.  A novel approach to the calculation of pothole-induced contact forces in MDOF vehicle models , 2004 .

[66]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[67]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[68]  Li Meng,et al.  Subpixel Motion Estimation for Super-Resolution Image Sequence Enhancement , 1998, J. Vis. Commun. Image Represent..

[69]  Emily Chen Shao Detecting sources of heat loss in residential buildings from infrared imaging , 2011 .

[70]  David A Bluemke,et al.  Spatial and temporal resolution in cardiovascular MR imaging: review and recommendations. , 2005, Radiology.

[71]  Constantinos A. Balaras,et al.  Infrared thermography for building diagnostics , 2002 .

[72]  Robert L. Stevenson,et al.  Spatial Resolution Enhancement of Low-Resolution Image Sequences A Comprehensive Review with Directions for Future Research , 1998 .

[73]  D. Dewitt,et al.  Theory and practice of radiation thermometry , 1988 .

[74]  Christian Raschner,et al.  An Overview of Recent Application of Medical Infrared Thermography in Sports Medicine in Austria , 2010, Sensors.

[75]  Isao Noda,et al.  Two-dimensional infrared spectroscopy , 1989 .

[76]  A. Ianiro,et al.  Temperature maps measurements on 3D surfaces with infrared thermography , 2012 .

[77]  Semiha Turkay,et al.  A Study of Random Vibration Characteristics of the Quarter-Car Model , 2004 .

[78]  Søren Østergaard Jensen,et al.  Validation of building energy simulation programs: a methodology , 1995 .

[79]  Frank E. Liebmann,et al.  Infrared calibration development at Fluke Corporation Hart Scientific Division , 2008, SPIE Defense + Commercial Sensing.

[80]  Shree K. Nayar,et al.  Ego-motion and omnidirectional cameras , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[81]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[82]  Robert P. Madding,et al.  Emissivity measurement and temperature correction accuracy considerations , 1999, Defense, Security, and Sensing.

[83]  M. J. Coleman,et al.  Radiometic Comparison Between a National Laboratory and an Industrial Laboratory , 2011 .

[84]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[85]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[86]  Gang Chen,et al.  Geospatial Technologies to Improve Urban Energy Efficiency , 2011, Remote. Sens..

[87]  Daniel E. Fisher,et al.  EnergyPlus: creating a new-generation building energy simulation program , 2001 .

[88]  G. Cristóbal,et al.  Simultaneous super-resolution and blind deconvolution , 2008 .

[89]  Yuan Jian A Fast Super-resolution Reconstruction Algorithm , 2006 .

[90]  Håkan Pettersson,et al.  Uncooled infrared bolometer arrays operating in a low to medium vacuum atmosphere: performance model and tradeoffs , 2007, SPIE Defense + Commercial Sensing.

[91]  J C Miñano Application of the conservation of etendue theorem for 2-D subdomains of the phase space in nonimaging concentrators. , 1984, Applied optics.

[92]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[93]  José A. Sobrino,et al.  Experimental measurements for studying angular and spectral variation of thermal infrared emissivity. , 2004, Applied optics.

[94]  Daniel F. Murphy,et al.  Microbolometer uncooled infrared camera with 20-mK NETD , 1998, Defense, Security, and Sensing.

[95]  K. Prazdny,et al.  Egomotion and relative depth map from optical flow , 2004, Biological Cybernetics.

[96]  George D. Skidmore,et al.  DRS uncooled VOx infrared detector development and production status , 2010, Defense + Commercial Sensing.

[97]  Robert L. Stevenson,et al.  Super-resolution from image sequences-a review , 1998, 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268).

[98]  X. Jordà,et al.  Irradiance-based emissivity correction in infrared thermography for electronic applications. , 2011, The Review of scientific instruments.

[99]  M. Wetter BuildOpt—a new building energy simulation program that is built on smooth models , 2005 .

[100]  Tony N.T. Lam,et al.  Principal component analysis and long-term building energy simulation correlation , 2010 .

[101]  David J. Fleet Measurement of image velocity , 1992 .

[102]  J. L. Tissot IR detection with uncooled focal plane arrays. State-of-the art and trends , 2004 .

[103]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[104]  M. Rencz,et al.  A thermal benchmark chip: design and applications , 1998 .

[105]  Jana Kosecka,et al.  Piecewise planar city 3D modeling from street view panoramic sequences , 2009, CVPR.

[106]  Michael H. Ramsey,et al.  Sampling as a source of measurement uncertainty: techniques for quantification and comparison with analytical sources , 1998 .

[107]  J. Clinch,et al.  Cost-benefit Analysis of Domestic Energy Efficiency , 2001 .

[108]  Mohammed Ghanbari,et al.  Scope of validity of PSNR in image/video quality assessment , 2008 .

[109]  Lawrence A. Bergman,et al.  POTHOLE-INDUCED CONTACT FORCES IN A SIMPLE VEHICLE MODEL , 2002 .

[110]  Zeke Hausfather,et al.  A High‐Resolution Statistical Model of Residential Energy End Use Characteristics for the United States , 2010 .

[111]  Michael Ray,et al.  320 x 240 microbolometer focal plane array for uncooled applications , 1996, Photonics West.

[112]  Palle Andersen,et al.  Modal Identification from Ambient Responses using Frequency Domain Decomposition , 2000 .

[113]  Alan Meier,et al.  Accuracy of home energy rating systems , 2000 .

[114]  Paul Fazio,et al.  A case study of validation of an energy analysis program: MICRO-DOE2.1E , 1998 .

[115]  Yehuda L. Klein,et al.  Residential energy conservation choices of poor households during a period of rising fuel prices , 1987 .

[116]  Kamal El Omari,et al.  2D-Infrared Thermography Monitoring of Ultrasound-Assisted Polymerization of Water-Soluble Monomer in a Gel Process , 2011 .

[117]  T. Chau,et al.  Infrared thermography as an access pathway for individuals with severe motor impairments , 2009, Journal of NeuroEngineering and Rehabilitation.