Artificial intelligence is a spectacular part of computer engineering that has earned a compelling diversion in the field of medical data classification due to its state-of-art algorithmic strength and learning capabilities. Machine Learning is a major sub-domain of artificial intelligence, where it has become one of the most promising fields in computer science. In recent years, there is a large spectrum of healthcare and biomedical data that has been growing intensely. Due to the huge labeled or unlabeled data, it is important to have a compact and robust machine learning solution for classification. Several optimizers have been deployed to improve the inclusive performance of machine learning models. The classification of machine learning models depends on several factors. This comprehensive review paper aims to insight into the current stage of optimized machine learning success on medical data classification. An increasing number of unstructured medical data has been utilizing in machine learning algorithms to predict intuitions. But it is difficult to inherent immense intuition from those data. So machine learning researchers have utilized state-of-art optimizers and novel feature selection techniques to overcome and emend the performance accuracy. We have highlighted some recent literature, which exhibits the robust impact of optimizers and feature selection on machine learning techniques on medical data characterization. On the other hand, a clean-cut introduction on machine learning and theoretical outlook of widely utilized optimization techniques like genetic algorithm, gray wolf optimization, and particle swarm optimization are discussed for initial understanding of the optimization techniques.