AGI imagined: social AGI and the co-formation of human-AGI society.

The AI industry is advancing toward an Artificial General Intelligence (AGI) that will possess cognitive capabilities that are equal to or stronger than those of humans. While a purely computational AGI may be cognitively powerful, it will probably struggle to understand the human social environment. This gap needs both a theoretical and an applicative solution. This article provides the theoretical solution, specifying what such an AGI must contain and how it must be formed. We propose a theoretical construct of an AGI that will not only be computational but will also be capable to develop internal social understanding, internalize norms in a manner similar to humans, and act upon this social understanding. We call such AGI a "Social AGI." We argue that Social AGI cannot be developed only by using technical alignment or external governance. Instead, it must undergo a staged development in which visible social behavior gradually becomes social cognition and norm internalization. This process cannot be separated from the parallel support that human society must provide the effort of developing Social AGI. While AI systems may become more familiar and embedded in institutions and society itself, the public perceptions of risk will probably decline. This normalization will provide the basis for granting the Social AGI progressively higher levels of autonomy. In order to capture this process of mutual change, we develop a socio-technical concept of human-AGI co-formation. We differentiate co-formation from co-existence, which describes humans and AI operating side by side, and from co-evolution, which describes reciprocal and cyclic influence. Co-formation refers to a process in which at the beginning Social AGI is shaped exclusively by human society. However, as the process progresses, they shape together the basic conditions under which the AGI is developed. On the AGI side, this will require the capabilities to internalize norms, and to be able to function according to its roles. On the societal side, this will require to create several mechanisms of role assignment, training and socializing the AGI, and monitoring its value alignment. The most important characteristics society should develop is a public acceptance and justification for increasing the AGI's autonomy. Society may learn not only how to supervise the AGI, but also it may learn to trust it.