In this work, we utilize a combination of free-text and structured data to build Acute Respiratory Distress Syndrome(ARDS) prediction models and ARDS phenotype clusters. We derived 'Patient Context Vectors' representing patientspecific contextual ARDS risk factors, utilizing deep-learning techniques on ICD and free-text clinical notes data. The Patient Context Vectors were combined with structured data from the first 24 hours of admission, such as vital signs and lab results, to build an ARDS patient prediction model and an ARDS patient mortality prediction model achieving AUC of 90.16 and 81.01 respectively. The ability of Patient Context Vectors to summarize patients' medical history and current conditions is also demonstrated by the automatic clustering of ARDS patients into clinically meaningful phenotypes based on comorbidities, patient history, and presenting conditions. To our knowledge, this is the first study to successfully combine free-text and structured data, without any manual patient risk factor curation, to build real-time ARDS prediction models.