Delphi: Towards Machine Ethics and Norms

Failing to account for moral norms could notably hinder AI systems’ ability to interact with people. AI systems empirically require social, cultural, and ethical norms to make moral judgments. However, open-world situations with different groundings may shift moral implications significantly. For example, while “driving my friend to the airport” is “good”, “driving my friend to the airport with a car I stole” is “not okay.” In natural language processing, machine moral reasoning is still in a preliminary stage, illuminating the importance of research on steering machines to making ethical judgments. Inspired by descriptive ethics, a line of research on morality focusing on people’s moral judgments relevant to everyday situations, we conduct the first major attempt to computationally explore the vast space of moral implications in real-world settings. We introduce COMMONSENSE NORM BANK, a semiautomatically constructed dataset from several sources (e.g., SOCIAL CHEMISTRY) with 1.7M instances of descriptive ethics, covering a wide spectrum of everyday situations in contextualized, narrative, and sociallyor demographicallybiased settings. We present Delphi, a unified model of descriptive ethics empowered by diverse data of people’s moral judgment from COMMONSENSE NORM BANK. Delphi is robust to generate categorical and/or open-text moral judgments (e.g., “it’s dangerous”) for complex real-life situations (e.g., “driving my friend to the airport early in the morning when I was drunk last night”). Delphi demonstrates highly promising empirical results, with 92.1% accuracy, which outperforms the out-ofthe-box GPT-3 model with extensive prompting by a significant margin (83.9%) . We also provide careful study of Delphi’s limitations, particularly with respect to undesirable biases against underrepresented population, opening doors to further investigation in future research in computational moral reasoning. Closing the gap between machines and people’s moral reasoning is a prerequisite for trustworthy open-world AI deployments. Moral judgment is never simplistic as there can be clash of different ethical/cultural values at play. Thus, developing high-quality corpus of people’s ethical judgment over diverse scenarios is needed to teach machines to make moral judgment. With optimistic promises demonstrated by Delphi, we inspire significant future research in this next frontier of AI, to facilitate reliable, socially aware, and ethically-informed future AI practices.