Monitoring human emotions through wearable systems has become an important area of research. Electrodermal activity (EDA) has proven to be a good indicator of emotional arousal, and numerous works have focused on using EDA data to predict emotional states. However, to successfully integrate EDA data into real-time wearable emotion recognition systems, several challenges of practical real-life scenarios, need to be addressed. This paper explores the relationship between speech signals and EDA reactions and analyzes a new approach for classification of skin conductance reactions elicited by emotional arousal using speech signals as a triggering event. Results show an average improvement in skin conductance reaction classification accuracy of at least 5.6% when using speech-triggered reactions compared with traditional methods. The use of speech as a triggering event could help improve real-time emotion recognition algorithms implemented within wearable systems.