214 12 The Engineering and Development of Ethics

214 12 The Engineering and Development of Ethics thinking, this is a cognitive precedent to being able to reconsider an entire belief system, one that has had contractual logic built atop reflexive adherence that began in early development. If the initial moral system is viewed as positive and stable, then this cognitive capacity is seen as dangerous and scary, but if early morality is stunted or warped, then this ability is seen as enlightened. However, achieving this cognitive stage does not mean one automatically changes their belief systems, but rather that the mental machinery is in place to consider the possibilities. Because many people do not reach this level of cognitive development in the area of moral and ethical thinking, it is associated with negative traits (“moral relativism” and “flip-flopping”). However, this cognitive flexibility generally leads to more sophisticated and applicable moral codes, which in turn leads to morality which is actually more stable because it is built upon extensive and deep consideration rather than simple adherence to reflexive or rationalized ideologies. 12.4.2 Stages of Development of Empathic Ethics Complementing Kohlberg’s logic-and-justice-focused approach, Carol Gilligan’s [Gil82] “ethics of care” model is a moral development theory which posits that empathetic understanding plays the central role in moral progression from an initial self-centered modality to a socially responsible one. The ethics of care model is concerned with the ways in which an individual cares (responds to dilemmas using empathetic responses) about self and others. As shown in Table 12.3, the ethics of care is broken into the same three primary stage as Kohlberg, but with a focus on empathetic, emotional caring rather than rationalized, logical principles of justice. Stage Principle of Care Pre-Conventional Individual Survival Conventional Self Sacrifice for the Greater Good Post-Conventional Principle of Nonviolence (do not hurt others, or oneself) Table 12.3: Gilligan’s Stages of the Ethics of Care For an “ethics of care” approach to be applied in an AGI, the AGI must be capable of internal simulation of other minds it encounters, in a similar manner to how humans regularly simulate one another internally. Without any mechanism for internal simulation, it is unlikely that an AGI can develop any sort of empathy toward other minds, as opposed to merely logically or probabilistically modeling other agents’ behavior or other minds’ internal contents. In a CogPrime context, this ties in closely with how CogPrime handles episodic knowledge — partly via use of an internal simulation world, which is able to play “mental movies” of prior and hypothesized scenarios within the AGI system’s mind. However, in humans empathy involves more than just simulation, it also involves sensorimotor responses, and of course emotional responses — a topic we will discuss in more depth in Appendix ?? where we review the functionality of mirror neurons and mirror systems in the human brains. When we see or hear someone suffering, this sensory input causes motor responses in us similar to if we were suffering ourselves, which initiates emotional empathy and corresponding cognitive processes. HOUSE_OVERSIGHT_013130