210 12 The Engineering and Development of Ethics

210 12 The Engineering and Development of Ethics We realize this point may be somewhat contentious — a counter-argument would be that the human brain is known to support at least moderately ethical behavior, according to human ethical standards, whereas less brain-like AGI systems are much less well understood. However, the obvious counter-counterpoints are that: e Humans are not all that consistently ethical, so that creating AGI systems potentially much more practically powerful than humans, but with closely humanlike ethical, motivational and goal systems, could in fact be quite dangerous e The effect on a human-like ethical/motivational/goal system of increasing the intelligence, or changing the physical embodiment or cognitive capabilities, of the agent containing the system, is unknown and difficult to predict given all the complexities involved The course we tentatively recommend, and are following in our own work, is to develop AGI systems with explicit, hierarchically-dominated goal systems. That is: e create one or more "top goals" (we call them Ubergoals in CogPrime ) e have the system derive subgoals from these, using its own intelligence, potentially guided by educational interaction or explicit programming e have a significant percentage of the system’s activity governed by the explicit pursuit of these goals Note that the "significant percentage" need not be 100%; CogPrime, for example, combines explicitly goal-directed activity with other "spontaneous" activity. Requiring that all activity be explicitly goal-directed may be too strict a requirement to place on AGI architectures. The next step, of course, is for the top-level goals to be chosen in accordance with the principle of human-Friendliness. The next one of our eight points, about the Global Brain, addresses one way of doing this. In our near-term work with CogPrime, we are using simplistic approaches, with a view toward early-stage system testing. 12.4 Ethical Synergy An explicit goal system provides an explicit way to ensure that ethical principles (as represented in system goals) play a significant role in guiding an AGI system’s behavior. However, in an integrative design like CogPrime the goal system is only a small part of the overall story, and it’s important to also understand how ethics relates to the other aspects of the cognitive architecture. One of the more novel ideas presented in this chapter is that different types of ethical intuition may be associated with different types of memory — and to possess mature ethics, a mind must display ethical synergy between the ethical processes associated with its memory types. Specifically, we suggest that: e Episodic memory corresponds to the process of ethically assessing a situation based on similar prior situations e Sensorimotor memory corresponds to “mirror neuron” type ethics, where you feel another person’s feelings via mirroring their physiological emotional responses and actions e Declarative memory corresponds to rational ethical judgment HOUSE_OVERSIGHT_013126