Big data mining using supervised machine learning approaches for Hadoop with Weka distribution

Data is increasing very rapidly with the increase in technologies. To process this data and performing accurate mining to yield conclusions is a challenge. This domain to process and mining this big data is termed as big data mining. To store and process big data many open source tools were proposed and are present in Apache foundation. Apache Hadoop is the most widely used tool for big data processing. Apache Hadoop consists of two main components namely Hadoop distributed file system (HDFS) and map/reduce. HDFS is used to store the data in distributed form and map/reduce is used to process this distributed spread data. In the past many data mining and classification approaches have been proposed for big data in which for implementing machine learning no standard tool is used. And no generic topology for data flow is proposed to implement such model. And accuracy of classification for raw dataset is also poor. In this dissertation to perform big data mining Apache Hadoop and Weka is used. Weka is an open source tool for machine learning proposed by Waikato university of New Zealand. Here in this work Apache Hadoop is connected with Weka. Using this combination big data is stored on HDFS and processed using Weka using Knowledge flow of Weka. Knowledge flow provides a means to construct topologies using them HDFS components can be used to provide data to machine learning algorithms provided in Weka. In this work supervised machine learning approaches which include Naïve Bayes, Support vector machine, J48 are used for big data mining. The accuracy of these approaches is compared for raw data and normalized data given to the same topology. It is found proposed approach for big data mining yields better results as compared to the reference approach.