WiForce: Wireless Sensing and Localization of Contact Forces on a Space Continuum

Contact force is a natural way for humans to interact with the physical world around us. However, most of our interactions with the digital world are largely based on a simple binary sense of touch (contact or no contact). Similarly, when interacting with robots to perform complex tasks, such as surgery, richer force information that includes both magnitude and contact location is important for task performance. To address these challenges, we present the design and fabrication of WiForce which is a ‘wireless’ sensor, sentient to contact force magnitude and location. WiForce achieves this by transducing force magnitude and location, to phase changes of an incident RF signal of a backscattering tag. The phase changes are thus modulated into the backscattered RF signal, which enables measurement of force magnitude and contact location by inferring the phases of the reflected RF signal. WiForce’s sensor is designed to support wide-band frequencies all the way up to 3 GHz. We evaluate the force sensing wirelessly in different environments, including through phantom tissue, and achieve force accuracy of 0.3 N and contact location accuracy of 0.6 mm.

[1]  Yang Hao,et al.  Antennas and Propagation of Implanted RFIDs for Pervasive Healthcare Applications , 2010, Proceedings of the IEEE.

[2]  Sachin Katti,et al.  Position Tracking for Virtual Reality Using Commodity WiFi , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  R. A. Davidoff Sensory Neurons , 1993, Neurology.

[4]  Ross T. Smith,et al.  Adding input controls and sensors to RFID tags to support dynamic tangible user interfaces , 2014, TEI '14.

[5]  Alanson P. Sample,et al.  A capacitive touch interface for passive RFID tags , 2009, 2009 IEEE International Conference on RFID.

[6]  James Chu,et al.  Microwave and RF Design , 2015 .

[7]  Fadel Adib,et al.  3D Backscatter Localization for Fine-Grained Robotics , 2019, NSDI.

[8]  Ting Yao,et al.  A droplet-based passive force sensor for remote tactile sensing applications , 2018 .

[9]  Javad Dargahi,et al.  Tactile Sensing and Displays: Haptic Feedback for Minimally Invasive Surgery and Robotics , 2012 .

[10]  Danica Kragic,et al.  Trends and challenges in robot manipulation , 2019, Science.

[11]  A. Todoroki,et al.  Wireless flexible capacitive sensor based on ultra-flexible epoxy resin for strain measurement of automobile tires , 2007 .

[12]  Jung Kim,et al.  Durable and Repairable Soft Tactile Skin for Physical Human Robot Interaction , 2017, HRI.

[13]  Li-Feng Wang,et al.  LC Passive Wireless Sensors Toward a Wireless Sensing Platform: Status, Prospects, and Challenges , 2016, Journal of Microelectromechanical Systems.

[14]  Haining Li,et al.  A Miniature Layered SAW Contact Stress Sensor for Operation in Cramped Metallic Slits , 2018, Instruments and Experimental Techniques.

[15]  Omid Salehi-Abari,et al.  In-body backscatter communication and localization , 2018, SIGCOMM.

[16]  Zhenan Bao,et al.  Biodegradable and flexible arterial-pulse sensor for the wireless monitoring of blood flow , 2019, Nature Biomedical Engineering.

[17]  Z. Jane Wang,et al.  Novel Tactile Sensor Technology and Smart Tactile Sensing Systems: A Review , 2017, Sensors.

[18]  Manos M. Tentzeris,et al.  Design of a highly sensitive wireless passive RF strain transducer , 2011, 2011 IEEE MTT-S International Microwave Symposium.

[19]  Suvranu De,et al.  Using the PhysX engine for physics‐based virtual surgery with force feedback , 2009, The international journal of medical robotics + computer assisted surgery : MRCAS.

[20]  C. Cao,et al.  Effect of haptic feedback in laparoscopic surgery skill acquisition , 2012, Surgical Endoscopy.

[21]  Hyosang Lee,et al.  Internal Array Electrodes Improve the Spatial Resolution of Soft Tactile Sensors Based on Electrical Resistance Tomography , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[22]  Sachin Katti,et al.  HitchHike: Practical Backscatter Using Commodity WiFi , 2016, SenSys.

[23]  Claudio Pacchierotti,et al.  Compensating for Fingertip Size to Render Tactile Cues More Accurately , 2020, IEEE Transactions on Haptics.

[24]  Albrecht Schmidt,et al.  Enabling implicit human computer interaction: a wearable RFID-tag reader , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[25]  Sachin Katti,et al.  FreeRider: Backscatter Communication Using Commodity Radios , 2017, CoNEXT.

[26]  Manuchehr Soleimani,et al.  Electrical Impedance Tomography for Artificial Sensitive Robotic Skin: A Review , 2015, IEEE Sensors Journal.

[27]  Balasundar I Raju,et al.  3-D finite-element models of human and monkey fingertips to investigate the mechanics of tactile sense. , 2003, Journal of biomechanical engineering.

[28]  Markus P. Nemitz,et al.  Soft Radio-Frequency Identification Sensors: Wireless Long-Range Strain Sensors Using Radio-Frequency Identification , 2019, Soft robotics.

[29]  Alex Olwal,et al.  FlexTiles: A Flexible, Stretchable, Formable, Pressure-Sensitive, Tactile Input Sensor , 2016, CHI Extended Abstracts.

[30]  D. Ginty,et al.  The Sensory Neurons of Touch , 2013, Neuron.

[31]  Nazim Haouchine,et al.  Vision-Based Force Feedback Estimation for Robot-Assisted Surgery Using Instrument-Constrained Biomechanical Three-Dimensional Maps , 2018, IEEE Robotics and Automation Letters.

[32]  Jihun Lee,et al.  A Distributed Wireless Network of Implantable Sub-mm Cortical Microstimulators for Brain-Computer Interfaces , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[33]  Jie Xiong,et al.  mD-Track: Leveraging Multi-Dimensionality for Passive Indoor Wi-Fi Tracking , 2018, MobiCom.

[34]  Edward H. Adelson,et al.  GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force , 2017, Sensors.

[35]  Hiroshi Ishii,et al.  The tangible user interface and its evolution , 2008, CACM.

[36]  J. R. Humphries,et al.  Passive, wireless SAW OFC strain sensor , 2012, 2012 IEEE International Frequency Control Symposium Proceedings.

[37]  Alicia Casals,et al.  Sensorless force estimation using a neuro-vision-based approach for robotic-assisted surgery , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[38]  Howie Choset,et al.  Continuum Robots for Medical Applications: A Survey , 2015, IEEE Transactions on Robotics.

[39]  Chang Liu,et al.  Recent Progress in Technologies for Tactile Sensors , 2018, Sensors.

[40]  Hyosang Lee,et al.  Calibrating a Soft ERT-Based Tactile Sensor with a Multiphysics Model and Sim-to-real Transfer Learning , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Yong-Lae Park,et al.  Design and Fabrication of Soft Artificial Skin Using Embedded Microchannels and Liquid Conductors , 2012, IEEE Sensors Journal.

[42]  Yang Zhang,et al.  Sozu: Self-Powered Radio Tags for Building-Scale Activity Sensing , 2019, UIST.

[43]  Yang Wang,et al.  Passive wireless smart-skin sensor using RFID-based folded patch antennas , 2011 .

[44]  Weiliang Xu,et al.  Haptic display for virtual reality: progress and challenges , 2019, Virtual Real. Intell. Hardw..

[45]  Gordon Cheng,et al.  Directions Toward Effective Utilization of Tactile Skin: A Review , 2013, IEEE Sensors Journal.

[46]  Sandeep K. S. Gupta,et al.  Towards a propagation model for wireless biomedical applications , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[47]  Alanson P. Sample,et al.  IDSense: A Human Object Interaction Detection System Based on Passive UHF RFID , 2015, CHI.

[48]  Tsu-Chin Tsao,et al.  The Effect of Haptic Feedback on Efficiency and Safety During Preretinal Membrane Peeling Simulation , 2019, Translational vision science & technology.

[49]  Qiongfeng Shi,et al.  Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications , 2020, Science Advances.

[50]  Ramachandran Ramjee,et al.  WiFi-Nano: reclaiming WiFi efficiency through 800 ns slots , 2011, MobiCom.

[51]  Arianna Menciassi,et al.  Finite Element Analysis and Design Optimization of a Pneumatically Actuating Silicone Module for Robotic Surgery Applications , 2014 .

[52]  Géry Casiez,et al.  ForceEdge: Controlling Autoscroll on Both Desktop and Mobile Computers Using the Force , 2017, CHI.

[53]  Karthikeyan Sundaresan,et al.  RIO: A Pervasive RFID-based Touch Gesture Interface , 2017, MobiCom.

[54]  Clifford R. Pollock,et al.  Stretchable distributed fiber-optic sensors , 2020, Science.

[55]  A. Castellanos,et al.  Force Feedback Plays a Significant Role in Minimally Invasive Surgery: Results and Analysis , 2005, Annals of surgery.

[56]  Alex S. Taylor,et al.  Rethinking RFID: awareness and control for interaction with RFID systems , 2010, CHI.

[57]  R. Dahiya,et al.  Soft eSkin: distributed touch sensing with harmonized energy and computing , 2019, Philosophical Transactions of the Royal Society A.

[58]  Ronald C Merrell,et al.  Training in laparoscopic suturing skills using a new computer-based virtual reality simulator (MIST-VR) provides results comparable to those with an established pelvic trainer system. , 2002, Journal of laparoendoscopic & advanced surgical techniques. Part A.

[59]  Ilana Nisky,et al.  Perception of Stiffness in Laparoscopy - the Fulcrum Effect , 2012, MMVR.

[60]  Michael C. Yip,et al.  Model-Less Hybrid Position/Force Control: A Minimalist Approach for Continuum Manipulators in Unknown, Constrained Environments , 2016, IEEE Robotics and Automation Letters.

[61]  Weiliang Xu,et al.  Toward Whole-Hand Kinesthetic Feedback: A Survey of Force Feedback Gloves , 2019, IEEE Transactions on Haptics.

[62]  Wei Chen,et al.  Tactile Sensors for Friction Estimation and Incipient Slip Detection—Toward Dexterous Robotic Manipulation: A Review , 2018, IEEE Sensors Journal.

[63]  A. Okamura Haptic feedback in robot-assisted minimally invasive surgery , 2009, Current opinion in urology.

[64]  Jianwei Zhang,et al.  Grasping Force Control of Multi-Fingered Robotic Hands through Tactile Sensing for Object Stabilization , 2020, Sensors.

[65]  K. J. Kuchenbecker,et al.  Surgeons and non-surgeons prefer haptic feedback of instrument vibrations during robotic surgery , 2015, Surgical Endoscopy.

[66]  D. Yuh,et al.  Effects of visual force feedback on robot-assisted surgical task performance. , 2008, The Journal of thoracic and cardiovascular surgery.

[67]  Benton H. Calhoun,et al.  A 150nW, 5ppm/o C, 100kHz On-Chip clock source for ultra low power SoCs , 2012, Proceedings of the IEEE 2012 Custom Integrated Circuits Conference.

[68]  Yousef Al-Handarish,et al.  A Survey of Tactile-Sensing Systems and Their Applications in Biomedical Engineering , 2020, Advances in Materials Science and Engineering.

[69]  Michael Haller,et al.  RESi: A Highly Flexible, Pressure-Sensitive, Imperceptible Textile Interface Based on Resistive Yarns , 2018, UIST.

[70]  Chen Li,et al.  Review of Research Status and Development Trends of Wireless Passive LC Resonant Sensors for Harsh Environments , 2015, Sensors.

[71]  Towards a Wireless Force Sensor Based on Wave Backscattering for Medical Applications , 2021, IEEE Sensors Journal.

[72]  Hyosang Lee,et al.  An ERT-based Robotic Skin with Sparsely Distributed Electrodes: Structure, Fabrication, and DNN-based Signal Processing , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[73]  S Leonhardt,et al.  Respiratory Monitoring System on the Basis of Capacitive Textile Force Sensors , 2011, IEEE Sensors Journal.

[74]  Michael Haller,et al.  Knitted RESi: A Highly Flexible, Force-Sensitive Knitted Textile Based on Resistive Yarns , 2020, SIGGRAPH Emerging Technologies.

[75]  Keith Winstein,et al.  Demo Abstract: RF Soil Moisture Sensing via Radar Backscatter Tags , 2020, 2020 19th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[76]  Patrick P. Mercier,et al.  20.1 A 28µW IoT Tag That Can Communicate with Commodity WiFi Transceivers via a Single-Side-Band QPSK Backscatter Communication Technique , 2020, 2020 IEEE International Solid- State Circuits Conference - (ISSCC).

[77]  Shwetak N. Patel,et al.  PaperID: A Technique for Drawing Functional Battery-Free Wireless Interfaces on Paper , 2016, CHI.

[78]  Yu She,et al.  Exoskeleton-covered soft finger with vision-based proprioception and tactile sensing , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[79]  Christopher R. Wagner,et al.  The role of force feedback in surgery: analysis of blunt dissection , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[80]  Xinyu Zhang,et al.  LiveTag: Sensing Human-Object Interaction Through Passive Chipless Wi-Fi Tags , 2019, GETMBL.

[81]  I. Dove,et al.  Analysis of Radio Propagation Inside the HumanBody for in-Body Localization Purposes , 2014 .

[83]  Ken Perlin,et al.  The UnMousePad: an interpolating multi-touch force-sensing input pad , 2009, SIGGRAPH 2009.