232 12 The Engineering and Development of Ethics

232 12 The Engineering and Development of Ethics Tweets and Youtube comments have their place in the cosmos, but they probably aren’t ideal in terms of helping humanity to form a coherent volition of some sort, suitable for providing an AGI with goal system guidance. A description of communication at the opposite end of the spectrum is presented in Adam Kahane and Peter Senge’s excellent book Solving Tough Problems [IkS04], which describes a methodology that has been used to reconcile deeply conflicting views in some very tricky real- world situations (e.g. helping to peacefully end apartheid in South Africa). One of the core ideas of the methodology is to have people with very different views explore different possible future scenarios together, in great detail — in cognitive psychology terms, a collective generation of hypothetical episodic knowledge. This has multiple benefits, including e emotional bonds and mutual understanding are built in the process of collaboratively ex- ploring the scenarios e the focus on concrete situations helps to break through some of the counterproductive abstract ideas that people (on both sides of any dichotomy) may have formed ® emergence of conceptual blends that might never have arisen only from people with a single point of view The result of such a process, when successful, is not an "average" of the participants views, but more like a "conceptual blend" of their perspectives. According to conceptual blending, which some hypothesize to be the core algorithm of cre- ativity [FT02], new concepts are formed by combining key aspects of existing concepts — but doing so judiciously, carefully choosing which aspects to retain, so as to obtain a high-quality and useful and interesting new whole. A blend is a compact entity that is similar to each of the entities blended, capturing their "essences" but also possessing its own, novel holistic integrity.... But in the case of blending different peoples’ world-views to form something new that everybody is going to have to live with (as in the case of finding a peaceful path beyond apartheid for South Africa, or arriving at. a humanity-wide CBV to use to guide an AGI goal system), the trick is that everybody has to agree that enough of the essence of their own view has been captured! This leads to the question of how to foster deep conceptual blending of diverse and divergent human perspectives, on a global scale. One possible answer is the creation of appropriate Global Brain oriented technologies — but moving away from technologies like Twitter that focus on quick and simple exchanges of small thoughts within affinity groups. On the face of it, it would seem what’s needed is just the opposite — long and deep exchanges of big concepts and deep feelings between individuals with radically different perspectives who would not commonly associate with each other. Building and effectively popularizing Internet technologies capable to foster this kind of interaction — quickly enough to be helpful with guiding the goal systems of the first highly powerful AGIs — seems a significant, though fascinating, challenge. Relationship with Coherent Extrapolated Volition The relation between this approach and CEV is interesting to contemplate. CEV has been loosely described as follows: "In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as HOUSE_OVERSIGHT_013148