Characterizing Everyday Objects using Human Touch: Thermal Dissipation as a Sensing Modality

We contribute MIDAS as a novel sensing solution for characterizing everyday objects using thermal dissipation. MIDAS takes advantage of the fact that anytime a person touches an object, it results in heat transfer. By capturing and modeling the dissipation of the transferred heat, e.g., through the decrease in the captured thermal radiation, MIDAS can characterize the object and determine its material. We validate MIDAS through extensive empirical benchmarks and demonstrate that MIDAS offers an innovative sensing modality that can recognize a wide range of materials – with up to 83% accuracy – and generalize to variations in the people interacting with objects.

[1]  Albrecht Schmidt,et al.  Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[2]  Alireza Sahami Shirazi,et al.  Investigation of Material Properties for Thermal Imaging-Based Interaction , 2015, CHI.

[3]  Eemil Lagerspetz,et al.  Evaluating Energy-Efficiency using Thermal Imaging , 2019, HotMobile.

[4]  Eemil Lagerspetz,et al.  Hot or Not? Robust and Accurate Continuous Thermal Imaging on FLIR cameras , 2019, 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom.

[5]  Rahul Arora,et al.  IoT based waste collection system using infrared sensors , 2016, INFOCOM 2016.

[6]  R L Klatzky,et al.  Identifying objects by touch: An “expert system” , 1985, Perception & psychophysics.

[7]  Jing Wang,et al.  The study of liquid surface waves with a smartphone camera and an image recognition algorithm , 2015 .

[8]  Ju Wang,et al.  TagScan: Simultaneous Target Imaging and Material Identification with Commodity RFID Devices , 2017, MobiCom.

[9]  Jorge Gonçalves,et al.  Probing Sucrose Contents in Everyday Drinks Using Miniaturized Near-Infrared Spectroscopy Scanners , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[10]  Rong Zhu,et al.  Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition , 2020, Science Robotics.

[11]  Romit Roy Choudhury,et al.  LiquID: A Wireless Liquid IDentifier , 2018, MobiSys.

[12]  Naser Hossein Motlagh,et al.  PENGUIN: aquatic plastic pollution sensing using AUVs , 2020, DroNet@MobiSys.

[13]  Mahantapas Kundu,et al.  Classification of Polar-Thermal Eigenfaces using Multilayer Perceptron for Human Face Recognition , 2008, 2008 IEEE Region 10 and the Third international Conference on Industrial and Information Systems.

[14]  Dina Katabi,et al.  Liquid Testing with Your Smartphone , 2019, MobiSys.

[15]  Tao Wang,et al.  Auracle: Detecting Eating Episodes with an Ear-mounted Sensor , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[16]  Bogdan Raducanu,et al.  Automatic Image-Based Waste Classification , 2019, IWINAC.

[17]  Jorge Gonçalves,et al.  Assisted Medication Management in Elderly Care Using Miniaturised Near-Infrared Spectroscopy , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[18]  Nicolai Marquardt,et al.  Deep Thermal Imaging: Proximate Material Type Recognition in the Wild through Deep Learning of Spatial Surface Temperature Patterns , 2018, CHI.

[19]  M. Acree.  Is there a gender difference in fingerprint ridge density? , 1999, Forensic science international.

[20]  Florian Alt,et al.  Stay Cool! Understanding Thermal Attacks on Mobile-based User Authentication , 2017, CHI.

[21]  Eric C. Larson,et al.  HeatWave: thermal imaging for surface user interaction , 2011, CHI.

[22]  Sasu Tarkoma,et al.  Toward Large-Scale Autonomous Marine Pollution Monitoring , 2021, IEEE Internet of Things Magazine.

[23]  Murat Haciomeroglu,et al.  Classification of TrashNet Dataset Based on Deep Learning Models , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[24]  Unsoo Ha,et al.  Learning Food Quality and Safety from Wireless Stickers , 2018, HotNets.

[25]  Massoud Pedram,et al.  Therminator: A thermal simulator for smartphones producing accurate chip and skin temperature maps , 2014, 2014 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED).

[26]  A. Hajdu,et al.  Detecting the excessive activation of the ciliaris muscle on thermal images , 2011, 2011 IEEE 9th International Symposium on Applied Machine Intelligence and Informatics (SAMI).

[27]  Andrea Bianchi,et al.  SpeCam: sensing surface color and material with the front-facing camera of a mobile device , 2017, MobileHCI.

[28]  Ko Nishino,et al.  Recognizing Material Properties from Images , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Guanghao Sun,et al.  A portable screening system for onboard entry screening at international airports using a microwave radar, reflective photo sensor and thermography , 2011, 2011 2nd International Conference on Instrumentation, Communications, Information Technology, and Biomedical Engineering.

[30]  Z. Obermeyer,et al.  Individual differences in normal body temperature: longitudinal big data analysis of patient records , 2017, British Medical Journal.

[31]  T. Walker,et al.  COVID-19 pandemic repercussions on the use and management of plastics. , 2020, Environmental science & technology.

[32]  Ali Akbar Babaei,et al.  Household recycling knowledge, attitudes and practices towards solid waste management , 2015 .