Open Source Computer Vision-based Layer-wise 3D Printing Analysis

The paper describes an open source computer vision-based hardware structure and software algorithm, which analyzes layer-wise the 3-D printing processes, tracks printing errors, and generates appropriate printer actions to improve reliability. This approach is built upon multiple-stage monocular image examination, which allows monitoring both the external shape of the printed object and internal structure of its layers. Starting with the side-view height validation, the developed program analyzes the virtual top view for outer shell contour correspondence using the multi-template matching and iterative closest point algorithms, as well as inner layer texture quality clustering the spatial-frequency filter responses with Gaussian mixture models and segmenting structural anomalies with the agglomerative hierarchical clustering algorithm. This allows evaluation of both global and local parameters of the printing modes. The experimentally-verified analysis time per layer is less than one minute, which can be considered a quasi-real-time process for large prints. The systems can work as an intelligent printing suspension tool designed to save time and material. However, the results show the algorithm provides a means to systematize in situ printing data as a first step in a fully open source failure correction algorithm for additive manufacturing.

[1]  Nicole E. Zander Recycled Polymer Feedstocks for Material Extrusion Additive Manufacturing , 2019, ACS Symposium Series.

[2]  K. Hart,et al.  Recycling meal-ready-to-eat (MRE) pouches into polymer filament for material extrusion additive manufacturing , 2018 .

[3]  Joshua M. Pearce,et al.  Reversing the Trend of Large Scale and Centralization in Manufacturing: The Case of Distributed Manufacturing of Customizable 3-D-Printable Self-Adjustable Glasses , 2014 .

[4]  Rhys Jones,et al.  RepRap – the replicating rapid prototyper , 2011, Robotica.

[5]  Joshua M. Pearce,et al.  Technical pathways for distributed recycling of polymer composites for distributed manufacturing: Windshield wiper blades , 2020, Resources, Conservation and Recycling.

[6]  U. Rajendra Acharya,et al.  A Novel Algorithm for Breast Lesion Detection Using Textons and Local Configuration Pattern Features With Ultrasound Imagery , 2019, IEEE Access.

[7]  Song-Chun Zhu,et al.  What are Textons? , 2005, Int. J. Comput. Vis..

[8]  Wojciech Matusik,et al.  MultiFab , 2015, ACM Trans. Graph..

[9]  Joshua M. Pearce,et al.  Distributed manufacturing with 3-D printing: a case study of recreational vehicle solar photovoltaic mounting systems , 2015 .

[10]  Nicole E. Zander,et al.  Recycled polypropylene blends as novel 3D printing materials , 2019, Additive Manufacturing.

[11]  Emanuele Menegatti,et al.  Different Approaches for Extracting Information from the Co-Occurrence Matrix , 2013, PloS one.

[12]  Catarina Mota,et al.  The rise of personal fabrication , 2011, C&C '11.

[13]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[14]  Yaser M. Banadaki,et al.  Smart additive manufacturing empowered by a closed-loop machine learning algorithm , 2019, Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[15]  A. Goshtasby Similarity and Dissimilarity Measures , 2012 .

[16]  Janne Heikkilä,et al.  Geometric Camera Calibration Using Circular Control Points , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Sung-Hyuk Cha Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions , 2007 .

[18]  Jens P. Wulfsberg,et al.  On the Economic Value of Open Source Hardware – Case Study of an Open Source Magnetic Resonance Imaging Scanner , 2019 .

[19]  Margaret Gillan,et al.  Recycled Cellulose Polypropylene Composite Feedstocks for Material Extrusion Additive Manufacturing , 2019, ACS omega.

[20]  Ian Parberry,et al.  3D Math Primer for Graphics and Game Development, 2nd Edition , 2011 .

[21]  BowyerAdrian,et al.  3D Printing and Humanity's First Imperfect Replicator , 2014 .

[22]  Joshua M. Pearce,et al.  Distributed Manufacturing of Flexible Products: Technical Feasibility and Economic Viability , 2017, Technologies.

[23]  Joshua M. Pearce,et al.  Fused Particle Fabrication 3-D Printing: Recycled Materials’ Optimization and Mechanical Properties , 2018, Materials.

[24]  Zhizhou Zhang,et al.  Autonomous in-situ correction of fused deposition modeling printers using computer vision and deep learning , 2019, Manufacturing Letters.

[25]  Song-Chun Zhu,et al.  What are Textons? , 2005, International Journal of Computer Vision.

[26]  Lucia L. Prieto-Godino,et al.  Open Labware: 3-D Printing Your Own Lab Equipment , 2015, PLoS biology.

[27]  Jochen Gehrig,et al.  Multi-template matching: a versatile tool for object-localization in microscopy images , 2019, BMC Bioinformatics.

[28]  Joshua M. Pearce,et al.  Emergence of Home Manufacturing in the Developed World: Return on Investment for Open-Source 3-D Printers , 2017 .

[29]  Andrew Zisserman,et al.  Texture classification: are filter banks necessary? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  Hans Burkhardt,et al.  Image classification using cluster cooccurrence matrices of local relational features , 2006, MIR '06.

[31]  Vincent Lepetit,et al.  Robust 3D Tracking with Descriptor Fields , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Geoffrey J. McLachlan,et al.  The EM Algorithm , 2012 .

[33]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[34]  Guangjun Zhang,et al.  Adaptive control of deposited height in GMAW-based layer additive manufacturing , 2014 .

[35]  Sergei Vassilvitskii,et al.  Worst-case and Smoothed Analysis of the ICP Algorithm, with an Application to the k-means Method , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[36]  Adrian Bowyer,et al.  RepRap: The Replicating Rapid Prototyper: Maximizing Customizability by Breeding the Means of Production , 2010 .

[37]  Joshua M. Pearce,et al.  Environmental Life Cycle Analysis of Distributed Three-Dimensional Printing and Conventional Manufacturing of Polymer Products , 2013 .

[38]  Michael Collins,et al.  EM Algorithm , 2010, Encyclopedia of Machine Learning.

[39]  Paulo Martins Engel,et al.  A Fast Incremental Gaussian Mixture Model , 2015, PloS one.

[40]  Krzysztof Okarma,et al.  Texture Based Quality Assessment of 3D Prints for Different Lighting Conditions , 2016, ICCVG.

[41]  Donald A. Adjeroh,et al.  Comparison of Texture Analysis Schemes Under Nonideal Conditions , 2011, IEEE Transactions on Image Processing.

[42]  Matti Pietikäinen,et al.  From BoW to CNN: Two Decades of Texture Representation for Texture Classification , 2018, International Journal of Computer Vision.

[43]  Joshua M. Pearce,et al.  Factors effecting real-time optical monitoring of fused filament 3D printing , 2017 .

[44]  Wen Gao,et al.  Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[45]  Christopher J. Sutcliffe,et al.  Automatic fault detection for laser powder-bed fusion using semi-supervised machine learning , 2019, Additive Manufacturing.

[46]  Joshua M. Pearce,et al.  Tightening the loop on the circular economy: Coupled distributed recycling and manufacturing with recyclebot and RepRap 3-D printing , 2018 .

[47]  Jefersson Alex dos Santos,et al.  Optical Flow Co-occurrence Matrices: A novel spatiotemporal feature descriptor , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[48]  Joshua M. Pearce Return on investment for open source scientific hardware development , 2016 .

[49]  Heinz Hügli,et al.  Fast ICP Algorithms for Shape Registration , 2002, DAGM-Symposium.

[50]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[51]  Jack Beuth,et al.  Anomaly Detection and Classification in a Laser Powder Bed Additive Manufacturing Process using a Trained Computer Vision Algorithm , 2018 .

[52]  Suphunnika Ibbotson,et al.  Direct digital manufacturing: definition, evolution, and sustainability implications , 2015 .

[53]  Shing I. Chang,et al.  Automated Process Monitoring in 3D Printing Using Supervised Machine Learning , 2018 .

[54]  Mark A Tschopp,et al.  Using Similarity Metrics to Quantify Differences in High-Throughput Data Sets: Application to X-ray Diffraction Patterns. , 2017, ACS combinatorial science.

[55]  Yili Hong,et al.  Profile monitoring based quality control method for fused deposition modeling process , 2018, Journal of Intelligent Manufacturing.

[56]  Joshua M. Pearce,et al.  Three Hundred and Sixty Degree Real-Time Monitoring of 3-D Printing Using Computer Analysis of Two Camera Views , 2017 .

[57]  Ben J. A. Kröse,et al.  Efficient Greedy Learning of Gaussian Mixture Models , 2003, Neural Computation.

[58]  Mauricio Camargo,et al.  Plastic recycling in additive manufacturing: A systematic literature review and opportunities for the circular economy , 2020, Journal of Cleaner Production.

[59]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[60]  Paulo Martins Engel,et al.  Correction: A Fast Incremental Gaussian Mixture Model , 2015, PloS one.

[61]  Jia Liu,et al.  Online Real-Time Quality Monitoring in Additive Manufacturing Processes Using Heterogeneous Sensors , 2015 .

[62]  Joshua M. Pearce Open-Source Lab: How to Build Your Own Hardware and Reduce Research Costs , 2013 .

[63]  Matti Pietikäinen,et al.  Computer Vision Using Local Binary Patterns , 2011, Computational Imaging and Vision.

[64]  Neil Gershenfeld,et al.  FAB: The Coming Revolution on Your Desktop--from Personal Computers to Personal Fabrication , 2005 .

[65]  Gabriela Oana Cula,et al.  Automatic Localization of Skin Layers in Reflectance Confocal Microscopy , 2014, ICIAR.

[66]  Eric Feron,et al.  3D Printing of a Leaf Spring: A Demonstration of Closed-Loop Control in Additive Manufacturing , 2018, 2018 IEEE Conference on Control Technology and Applications (CCTA).

[67]  Leyland Pitt,et al.  Disruptions, decisions, and destinations: enter the age of 3-D printing and additive manufacturing , 2015, IEEE Engineering Management Review.

[68]  Jitendra Malik,et al.  Contour and Texture Analysis for Image Segmentation , 2001, International Journal of Computer Vision.

[69]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[70]  Joshua M. Pearce,et al.  Wood furniture waste–based recycled 3-D printing filament , 2017 .

[71]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[72]  Jochen Gehrig,et al.  Multi-template matching: a versatile tool for object-localization in microscopy images , 2020, BMC Bioinformatics.

[73]  Du-Ming Tsai,et al.  Fast normalized cross correlation for defect detection , 2003, Pattern Recognit. Lett..

[74]  H. Edelsbrunner,et al.  Efficient algorithms for agglomerative hierarchical clustering methods , 1984 .

[75]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Joshua M. Pearce,et al.  Environmental Impacts of Distributed Manufacturing from 3-D Printing of Polymer Components and Products , 2013 .

[77]  Andrew Zisserman,et al.  A Statistical Approach to Texture Classification from Single Images , 2004, International Journal of Computer Vision.

[78]  Joshua M. Pearce,et al.  Multi-material additive and subtractive prosumer digital fabrication with a free and open-source convertible delta RepRap 3-D printer , 2015 .

[79]  A. Nassar,et al.  Intra-layer closed-loop control of build plan during directed energy additive manufacturing of Ti–6Al–4V , 2015 .

[80]  Joshua M. Pearce Quantifying the Value of Open Source Hardware Development , 2014, Modern Economy.

[81]  Lei Ren,et al.  Customized production based on distributed 3D printing services in cloud manufacturing , 2016 .

[82]  John D. Bernardin,et al.  A Framework for Additive Manufacturing Process Monitoring & Control , 2017 .

[83]  Fiona Charnley,et al.  Distributed manufacturing: scope, challenges and opportunities , 2016 .

[84]  Joshua M. Pearce,et al.  3-D Printing of Open Source Appropriate Technologies for Self-Directed Sustainable Development , 2010, Journal of Sustainable Development.

[85]  John Irwin,et al.  Life-cycle economic analysis of distributed manufacturing with open-source 3-D printers , 2013, Mechatronics.

[86]  Joshua M. Pearce,et al.  Impact of DIY Home Manufacturing with 3D Printing on the Toy and Game Market , 2017 .

[87]  Guang Yang,et al.  Automatic skin lesion segmentation by coupling deep fully convolutional networks and shallow network with textons , 2019, Journal of medical imaging.

[88]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[89]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[90]  Joshua M. Pearce,et al.  Building Research Equipment with Free, Open-Source Hardware , 2012, Science.

[91]  Sampsa Hyysalo,et al.  Anticipated environmental sustainability of personal fabrication , 2015 .

[92]  Sara McMains,et al.  Machine‐Learning‐Based Monitoring of Laser Powder Bed Fusion , 2018, Advanced Materials Technologies.

[93]  Eric Feron,et al.  Foundations of Intelligent Additive Manufacturing , 2017, ArXiv.