Optimal Control for Electromagnetic Haptic Guidance Systems

We introduce an optimal control method for electromagnetic haptic guidance systems. Our real-time approach assists users in pen-based tasks such as drawing, sketching or designing. The key to our control method is that it guides users, yet does not take away agency. Existing approaches force the stylus to a continuously advancing setpoint on a target trajectory, leading to undesirable behavior such as loss of haptic guidance or unintended snapping. Our control approach, in contrast, gently pulls users towards the target trajectory, allowing them to always easily override the system to adapt their input spontaneously and draw at their own speed. To achieve this flexible guidance, our optimization iteratively predicts the motion of an input device such as a pen, and adjusts the position and strength of an underlying dynamic electromagnetic actuator accordingly. To enable real-time computation, we additionally introduce a novel and fast approximate model of an electromagnet. We demonstrate the applicability of our approach by implementing it on a prototypical hardware platform based on an electromagnet moving on a bi-axial linear stage, as well as a set of applications. Experimental results show that our approach is more accurate and preferred by users compared to open-loop and time-dependent closed-loop approaches.

[1]  K. Sasaki,et al.  Learning to simplify , 2016, ACM Trans. Graph..

[2]  Saeid Nahavandi,et al.  Haptic handwriting aid for training and rehabilitation , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[3]  Otmar Hilliges,et al.  Contact-free Nonplanar Haptics with a Spherical Electromagnet , 2020, 2020 IEEE Haptics Symposium (HAPTICS).

[4]  Haruo Noma,et al.  Proactive Desk II: Development of a New Multi-object Haptic Display Using a Linear Induction Motor , 2006, IEEE Virtual Reality Conference (VR 2006).

[5]  D. Maynes-Aminzade,et al.  The actuated workbench: computer-controlled actuation in tabletop tangible interfaces , 2003, ACM Trans. Graph..

[6]  Otmar Hilliges,et al.  DeepWriting: Making Digital Ink Editable via Deep Generative Modeling , 2018, CHI.

[7]  Otmar Hilliges,et al.  Optimizing for aesthetically pleasing quadrotor camera motion , 2018, ACM Trans. Graph..

[8]  Alexander Domahidi,et al.  Real-time planning for automated multi-view drone cinematography , 2017, ACM Trans. Graph..

[9]  Jovan Popovic,et al.  Simulation of Human Motion Data using Short‐Horizon Model‐Predictive Control , 2008, Comput. Graph. Forum.

[10]  Bing-Yu Chen,et al.  Stylus assistant: designing dynamic constraints for facilitating stylus inputs on portable displays , 2016, SIGGRAPH ASIA Emerging Technologies.

[11]  Amit Zoran,et al.  Physical rendering with a digital airbrush , 2014, SIGGRAPH '14.

[12]  Yasuaki Kakehi,et al.  Comp*Pass: a compass-based drawing interface , 2014, CHI Extended Abstracts.

[13]  Karon E. MacLean,et al.  Phasking on Paper: Accessing a Continuum of PHysically Assisted SKetchING , 2020, CHI.

[14]  David Lindlbauer,et al.  Omni: Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback , 2020, UIST.

[15]  Geehyuk Lee,et al.  Digital rubbing: playful and intuitive interaction technique for transferring a graphic image onto paper with pen-based computing , 2008, CHI Extended Abstracts.

[16]  Sebastian Boring,et al.  Magnetips: Combining Fingertip Tracking and Haptic Feedback for Around-Device Interaction , 2019, CHI.

[17]  Jun Kato,et al.  Reactile: Programming Swarm User Interfaces through Direct Physical Manipulation , 2018, CHI.

[18]  Hiroshi Ishikawa,et al.  Mastering Sketching: Adversarial Augmentation for Structured Prediction , 2017 .

[19]  Karon E. MacLean,et al.  Predictive haptic guidance: intelligent user assistance for the control of dynamic tasks , 2006, IEEE Transactions on Visualization and Computer Graphics.

[20]  Suranga Nanayakkara,et al.  I-Draw: towards a freehand drawing assistant , 2014, OZCHI.

[21]  Joseph A. Paradiso,et al.  FreeD: a freehand digital sculpting tool , 2013, CHI.

[22]  Raffaello D'Andrea,et al.  A model predictive controller for quadrocopter state interception , 2013, 2013 European Control Conference (ECC).

[23]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[24]  Chris Manzie,et al.  Model predictive contouring control , 2010, 49th IEEE Conference on Decision and Control (CDC).

[25]  Joseph A. Paradiso,et al.  The Wise Chisel: The Rise of the Smart Handheld Tool , 2014, IEEE Pervasive Computing.

[26]  Mark Mulder,et al.  Haptic shared control: smoothly shifting control authority? , 2012, Cognition, Technology & Work.

[27]  Rolf Findeisen,et al.  Model predictive path-following for constrained nonlinear systems , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[28]  Wanjoo Park,et al.  Combining Full and Partial Haptic Guidance Improves Handwriting Skills Development , 2018, IEEE Transactions on Haptics.

[29]  Pedro Lopes,et al.  Muscle-plotter: An Interactive System based on Electrical Muscle Stimulation that Produces Spatial Output , 2016, UIST.

[30]  Manfred Morari,et al.  Optimization‐based autonomous racing of 1:43 scale RC cars , 2015, ArXiv.

[31]  Kar W. Yung,et al.  An Analytic Solution for the Force Between Two Magnetic Dipoles , 1998 .

[32]  Yasuaki Kakehi,et al.  dePENd: augmented handwriting system using ferromagnetism of a ballpoint pen , 2013, UIST.

[33]  Alex Olwal,et al.  shiftIO: Reconfigurable Tactile Elements for Dynamic Affordances and Mobile Interaction , 2017, CHI.

[34]  João Pedro Hespanha,et al.  Performance limitations in reference tracking and path following for nonlinear systems , 2008, Autom..

[35]  AN Kolmogorov-Smirnov,et al.  Sulla determinazione empírica di uma legge di distribuzione , 1933 .

[36]  Karl Johan Åström,et al.  PID Controllers: Theory, Design, and Tuning , 1995 .

[37]  Amit Zoran,et al.  D-Coil: A Hands-on Approach to Digital 3D Models Design , 2015, CHI.

[38]  Bruce P. Gibbs,et al.  Advanced Kalman Filtering, Least-Squares and Modeling: A Practical Handbook , 2011 .

[39]  Li-Yi Wei,et al.  Autocomplete hand-drawn animations , 2015, ACM Trans. Graph..

[40]  Chris Manzie,et al.  Model Predictive Contouring Control for Biaxial Systems , 2013, IEEE Transactions on Control Systems Technology.

[41]  Jan O. Borchers,et al.  FingerFlux: near-surface haptic feedback on tabletops , 2011, UIST.