Machine learning in a data-limited regime: Augmenting experiments with synthetic data uncovers order in crumpled sheets

Machine learning reveals order in crumpled sheets using simulated flat-folding patterns as data surrogate in a data-limited regime. Machine learning has gained widespread attention as a powerful tool to identify structure in complex, high-dimensional data. However, these techniques are ostensibly inapplicable for experimental systems where data are scarce or expensive to obtain. Here, we introduce a strategy to resolve this impasse by augmenting the experimental dataset with synthetically generated data of a much simpler sister system. Specifically, we study spontaneously emerging local order in crease networks of crumpled thin sheets, a paradigmatic example of spatial complexity, and show that machine learning techniques can be effective even in a data-limited regime. This is achieved by augmenting the scarce experimental dataset with inexhaustible amounts of simulated data of rigid flat-folded sheets, which are simple to simulate and share common statistical properties. This considerably improves the predictive power in a test problem of pattern completion and demonstrates the usefulness of machine learning in bench-top experiments where data are good but scarce.

[1]  Y. Pomeau,et al.  Crumpled paper , 1997, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[2]  J. Herskowitz,et al.  Proceedings of the National Academy of Sciences, USA , 1996, Current Biology.

[3]  Geoffrey E. Hinton,et al.  Deep Belief Networks for phone recognition , 2009 .

[4]  Paul W. Eastwick,et al.  Is Romantic Desire Predictable? Machine Learning Applied to Initial Romantic Attraction , 2017, Psychological science.

[5]  A. Hansen,et al.  Ridge network in crumpled paper. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[6]  Ullrich Köthe,et al.  Ilastik: Interactive learning and segmentation toolkit , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[7]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[8]  Daniel Rueckert,et al.  2011 8TH IEEE INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING: FROM NANO TO MACRO , 2011, ISBI 2011.

[9]  Paul Raccuglia,et al.  Machine-learning-assisted materials discovery using failed experiments , 2016, Nature.

[10]  E A Baltz,et al.  Achievement of Sustained Net Plasma Heating in a Fusion Experiment with the Optometrist Algorithm , 2017, Scientific Reports.

[11]  Chris H Rycroft,et al.  VORO++: a three-dimensional voronoi cell library in C++. , 2009, Chaos.

[12]  Bill Goodwine,et al.  A review of origami applications in mechanical engineering , 2016 .

[13]  T. Witten Stress focusing in elastic sheets , 2007 .

[14]  Xuchen Han,et al.  A material point method for thin shells with frictional contact , 2018, ACM Trans. Graph..

[15]  Prabhat,et al.  Deep Neural Networks for Physics Analysis on low-level whole-detector data at the LHC , 2017, Journal of Physics: Conference Series.

[16]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[17]  Roger G. Melko,et al.  Machine learning phases of matter , 2016, Nature Physics.

[18]  E. Sharon,et al.  Direct observation of the temporal and spatial dynamics during crumpling. , 2010, Nature materials.

[19]  C. Lintott,et al.  Galaxy Zoo: reproducing galaxy morphologies via machine learning★ , 2009, 0908.2033.

[20]  P. Baldi,et al.  Searching for exotic particles in high-energy physics with deep learning , 2014, Nature Communications.

[21]  C McCollin Applied stochastic models in business and industry , 2011 .

[22]  Christopher Wolverton,et al.  Accelerated discovery of metallic glasses through iteration of machine learning and high-throughput experiments , 2018, Science Advances.

[23]  Chris H. Rycroft Voro++: a three-dimensional Voronoi cell library in C++ , 2009 .

[24]  Sharon C Glotzer,et al.  Machine learning for crystal identification and discovery , 2017, 1710.09861.

[25]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[26]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[27]  P. N. Hobson,et al.  Engineering for profit from waste. Proceedings of the Institution of Mechanical Engineers , 1988 .

[28]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[29]  Andrea J. Liu,et al.  Relationship between local structure and relaxation in out-of-equilibrium glassy systems , 2016, Proceedings of the National Academy of Sciences.

[30]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[31]  Jan Hendrik Witte,et al.  Deep Learning for Finance: Deep Portfolios , 2016 .

[32]  W. Marsden I and J , 2012 .

[33]  C. Rycroft,et al.  A state variable for crumpled thin sheets , 2018, Communications Physics.

[34]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[35]  Ming C. Lin,et al.  Example-guided physically based modal sound synthesis , 2013, ACM Trans. Graph..

[36]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[37]  Kipp W. Johnson,et al.  Machine learning in cardiovascular medicine: are we there yet? , 2018, Heart.

[38]  Nicholas G. Polson,et al.  Deep learning for finance: deep portfolios: J. B. HEATON, N. G. POLSON AND J. H. WITTE , 2017 .

[39]  Geoffrey E. Hinton,et al.  Distilling a Neural Network Into a Soft Decision Tree , 2017, CEx@AI*IA.

[40]  Geoffrey Zweig,et al.  Recent advances in deep learning for speech research at Microsoft , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[41]  Neil Genzlinger A. and Q , 2006 .

[42]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[43]  Jennifer M. Rieser,et al.  Identifying structural flow defects in disordered solids using machine-learning methods. , 2014, Physical review letters.

[44]  Andrea J Liu,et al.  Disconnecting structure and dynamics in glassy thin films , 2016, Proceedings of the National Academy of Sciences.

[45]  Martin Wattenberg,et al.  Embedding Projector: Interactive Visualization and Interpretation of Embeddings , 2016, ArXiv.

[46]  J. Clune,et al.  The Surprising Creativity of Digital Evolution , 2018, ALIFE.

[47]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Dong Yu,et al.  Large vocabulary continuous speech recognition with context-dependent DBN-HMMS , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[49]  James F. O'Brien,et al.  Folding and crumpling adaptive sheets , 2013, ACM Trans. Graph..

[50]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.