Machine Learning: Data Pre-processing

In prognostics and health management (PHM), data pre‐processing generally involves the following tasks: data cleansing, normalization, feature discovery, and imbalanced data management. Data cleansing is the process of detecting and correcting corrupt or inaccurate data. Feature engineering is the process of using domain knowledge of the data to create features that make machine learning algorithms work. Feature extraction, also known as dimensionality reduction, is the transformation of high‐dimensional data into a meaningful representation of reduced dimensionality, which should have a dimensionality that corresponds to the intrinsic dimensionality of the data. Linear discriminant analysis (LDA) is commonly used as a dimensionality reduction technique in the data pre‐processing step for classification and machine learning applications. Feature selection, also called variable selection/attribute selection, is the process of selecting a subset of relevant features for use in model construction. The synthetic minority oversampling technique (SMOTE) algorithm produces artificial data based on the feature space similarities between minority data points.