On Forecasting Lung Cancer Patients’ Survival Rates Using 3D Feature Engineering

This thesis focuses on the application of Machine Learning in the healthcare domain, by extracting meaningful information from medical image reports using image processing techniques. Rather than considering a quintessential classification problem, however, we consider a regression problem whereby we aim to predict the survival times of patients after they have been diagnosed with adenocarcinoma, a type of lung cancer. We attempt to achieve this by processing chest CT scans to isolate the cancerous nodules, and extracting relevant features from the images of the nodules. Our work proposes the engineering of a feature set that can qualify the tumours with traits that are not visible to the naked eye using textural and statistical measures. We first consider the data in its 2D form, i.e., pixel data directly obtained from the images themselves, and create a benchmark using 2D Haralick computations. This feature set is then appended with statistical shape measurements as well. Furthermore, we offer two additional schemes in feature set generation, the first acting as a causal to the second. Taking the 2D feature set, we analyze the feature measurements in relation to the tumour depth as the tumour progresses through the slices of the scan. Cumulating this knowledge into a single measurement not only significantly improves the regression results, but also demonstrates the advantage of focusing on the prediction of short-term survival rate timelines as opposed to longterm survival rate timelines. The second scheme, founded by the results obtained by the first, concentrates on considering the cancer nodule in its 3D entirety, rather than just the images. This results in a considerably large feature space, with over 100 dimensions. To process these, we explore dimensionality reduction techniques, particularly data diagonalization in a block-diagonal matrix manner, to further enhance regression results.