Machine Ethics: What Matters to a Machine?

I argue that there is a gap between so-called “ethical reasoners” and “ethical-decision makers” that can’t be bridged by simply giving an ethical reasoner decision-making abilities. Ethical reasoning qua reasoning is distinguished from other sorts of reasoning mainly by being incredibly difficult, because it involves such thorny problems as analogical reasoning, and resolving conflicts among imprecise precepts. The ability to do ethical-decision making, however, requires knowing what an ethical conflict is, to wit, a clash between self-interest and what ethics prescribes. I construct a fanciful scenario in which a program could find itself in what seems like such a conflict, but argue that in any such situation the program’s “predicament” would not count as a real ethical conflict. Ethical decisions are those for which we feel tempted to cheat. The problem is not that computational agents could not have clashing desires, between obeying ethical principles and achieving “selfish” goals. I lay out plausible conditions under which we would be justified in saying that an agent was in exactly this position. The problem is that only a system with an architecture like the one evolution has burdened us with could suffer, and have to battle, temptation. Without temptation the domain of ethical reasoning is no different from any other possible application domain except that it is extremely difficult and unpromising.