Improving the Failure Understanding in Embedded Systems through Interactive Visual Support: An Evaluation Story

Effectively analyzing failure mechanisms is critical for ensuring safety and reliability aspects in embedded systems. This requires close collaborations between the system engineers who design the system and the safety experts who analyze the failures. To support this process, we present an interactive visual environment, called ESSAVis++, that helps in extracting the required safety information about the failure mechanisms in the underlying embedded system. In this paper, we focus mainly on the conducted user evaluation study to compare between ESSAVis++ and ESSaRel (a tool used in industry) in terms of accuracy, efficiency, and user acceptance. The goal was to determine the suitability and effectiveness of a common interactive visual environment that would be used by the system engineers and safety experts during the safety analysis process in embedded systems. Results indicate an increase in participants' efficiency and accuracy levels using the ESSAVis++ platform, due to the visual support and provided interaction options. We also observed a high acceptance rate among the participants towards ESSAVis++, mainly due to its intuitive visualizations.