Central Administration Complex, Prime City, Reisenburg, Sector W-7The Second for Simulations looked thoughtful. "Actually, now that you mention it, Geppetto did some very interesting work on them recently. It's difficult for me to summarize properly; I'm afraid I may have been too preoccupied in the theoretical papers he released along with the analysis..."
The First smiled sadly. "Well, it happens to the best of us. Perhaps we should invite him to discuss the matter? Rashid, do you know if he's been involved in anything that would stop him from sparing us the time?"
"Has he ever been?"
"True. It would be one for the record books if he were too busy to talk about his work. Why don't we take a momentary break while you put together the request for him, and we'll see what our resident master psychologist has to say about the Commune?"
February 3, 3400
Considerably Longer After Lunch
Geppetto had been created as a full Turing-compatible artificial general intelligence, specialized in the creation of advanced subsentient “expert systems” software for Umerian industrial and military operations. He had proceeded along these lines, gradually being augmented for faster operation as he proved exceptionally adept at the creation of new, efficient algorithms. Finally he was ‘promoted’ to the more challenging task of verifying friendliness in other full AIs; the limits of friendliness verification had always been one of the sharper bounds on the Technocracy’s development of advanced AIs; Geppetto soon proved to be a trustworthy and reliable guide to whether other AIs were themselves trustworthy and reliable.
Gradually, the computational intelligence established himself as a leading expert in the underlying theory of intelligence, composing publications and engaging in projects of ever increasing scope. His interests expanded to take in human and alien psychology as well as that of computers; his capabilities expanded as successful projects paid royalties that were rolled over into improved hardware.
Geppetto’s computational power had reached a comfortable plateau many decades ago. By human standards he had spent a tremendous amount of time considering the general nature of intelligence and the behavior of intelligent beings of all sorts. By the standards of AI’s, measuring time in processor cycles, that amount was nearly inconceivable. Geppetto was one of the five largest computational intelligences in the Technocracy. While not widely known outside Umerian borders save among highly specialized circles, an honest assessment would identify him as one of human space’s leading authorities on generalized psychology and sociology.
Indeed, he had once been offered the post of Second for Simulations, with excellent reasons; no one was entirely certain why he had seen fit to decline.
The screen remained black, but the room’s speakers went live, reproducing the resonant, cultured baritone Geppetto preferred to use as a voice, with careful diction and at a speed tuned to the comprehension of his audience.
“Good afternoon, ladies and gentlemen of the Council. You wish to speak with me regarding my recent analysis of the Commune?”
The First Technarch replied. “Ah, yes, Geppetto.” He had to suppress the impulse to add the honorific ‘doctor.’ In this case it would be extremely well-deserved, but the AI himself declined formal awards of academic titles as firmly as he did government posts. “I presume Dr. Ansary’s message was clear enough?”
“Dr. Ansary has informed me that the Commune has extended an offer of a mutual defense pact towards us, with comments implying that similar offers had been extended to the Interstellar Union of Worlds and the newly arrived Central Alliance. He has also been quite helpful in extending recordings of the relevant conversations, so far as they are available. You wish an evaluation of the Commune’s government, with an eye to whether a defense pact with them is likely to draw us into unnecessary and undesired conflict?”
“Well put, Geppetto, that is very much what we had in mind.”
“I’d thought so. I think the most salient result of my analysis is that, with p less than 0.0001, there is a massive centralized computational intelligence that has been strongly affecting Commune policy for at least the past several decades.
“The entity in question is not the sole organ of the Commune’s decision making process- I count at least three major independent decision-making nodes, more likely four. The Supreme Soviet remains a relevant factor, obviously the espionage branch is still in play, and... I believe there is a collective of AIs influencing the Commune’s military; possibly sentient warships, possibly fleet bases. Whether they constitute a separate node is not certain, though I deem it likely.
“But the computational intelligence identified by the latest round of analysis is, I think, the most powerful single node in the system, with chief responsibility for economic planning and considerable influence in other areas. I speculate that the directing entity has grown so powerful in terms of raw processing power that other elements of the Commune find themselves somewhat in awe of it. Even given that it is a very powerful computational intelligence, this speculated degree of awe would probably be greater than it deserves.”
Rashid looked curious. “Geppetto, usually you say ‘artificial’ intelligence...”
“Dr. Ansary, I’m sure you could infer my reasoning, but for the sake of clarity: I use the term ‘artificial’ to refer to designed intelligences.” There was a slight sniff. “With p less than 0.005, the directing entity behind recent Commune policy is not a product of intelligent design, there being no formal architecture to preserve its coherence. I infer that it is some kind of fusion, probably created on an ad hoc basis from various smaller minds. Hence, it is not an artifact in the sense that its structure reflects a truly rational design scheme. Computational, but not artificial.
“I emphasize that this directing entity is a very powerful fusion, probably composed of millions of separate minds, with total computational power rivalling or even exceeding that of the high-end Sovereignty AIs. But it is still a fusion, with all the limits on efficiency and stability that implies.”
Dr. Chernov frowned. “Unstable... are you implying some form of insanity?”
“A prudent question, difficult to answer without a rigorous definition of insanity. In some respects, yes: lack of personality integration is generally considered insanity, and it is, in principle, possible for this directing entity to develop delusions.”
That caused a stir. The prospect of the Commune’s considerable resources being taken over by a delusional moderately-godlike entity was... not something anyone wanted on the horizon.
“This is a low-order probability. The main source of the risk is that the fusion in question is most probably composed of many like-minded individuals, who all have essentially the same underlying assumptions about physical and social reality. Where those assumptions are sound, this is not a problem; where they are not, they tend to reinforce each other into an unassailable, interlocking fanaticism. You are all familiar with the nature and perils of ‘groupthink,’ but groupthink is a miserably inadequate word for the kind of difficulties that can arise among minds as intimately locked together as those of a fusion.”
“Thus, in this case, I project that the Commune’s directing entity is vulnerable to delusions to roughly the same extent to which the Commune’s dominant social theories diverge from reality in consequential ways. I am not ready to comment on those theories, save to observe that they are not formally verified to my own satisfaction. Though I am cautiously optimistic of being able to compare them against some general-case theoretical work I’ve been doing on the side before the century is out.”
The AI paused- obviously a metered one, for effect; no communication Geppetto ever made was unconsidered.
“In practice, I do think there is a significant risk of the Commune drawing itself and its allies into unnecessary conflicts, and that this must be considered in any prospective alliance. On the other hand, much depends on exactly what the Commune would be proposing an alliance against. A straightforward defensive alliance, such as the one we have with the NenAltKik, or the one I suspect is now being contemplated with... Tianguo?”
Geppetto was trusted enough that Dr. O’Connell felt no hesitation in saying “The French Empire, actually.”
“Ah, I see. By the way, let me congratulate you on the choice to add a metacognitive to your ranks; it makes the Council far more difficult to model.”
“Thank you, Geppetto. You were saying?”
“A straightforward defensive alliance, such as the one we have with the NenAltKik, might be imprudent with the Commune. They are potentially a powerful ally, but the gap in mindsets is great. In particular, Umeria has no equivalent to their... “Mirage” in institutional attitude, and nothing remotely analogous to the directing CI fusion I mentioned earlier. Even I would be extremely hard-pressed to estimate what course such a powerful and fragmented mind might take, and could never say with any real confidence.
“Therefore, I submit that the Commune is likely to try to manipulate the international scene in ways informed by their dominant social theory of Marxism. Since their reasoning is likely to be obscure to us, their actions would be unpredictable in the extreme. To some extent they may have already begun to make such manipulations without our knowledge. This makes a conventional alliance with them hazardous, as it greatly increases the risk that we will be drawn into an unwanted and unjustified war because of their actions in a third party’s territory.
“On the other hand, this alliance proposal was originally made in light of the threat of hostile moderately or strongly-godlike entities. And that is a very different matter. Against unusual or exceptionally powerful godlike threats, the Commune would be a very useful friend to have. They are technologically advanced, with an unusual technical palette that gives them many interesting options. They are resistant to physical attack and, with their social doctrines, also to memetic attacks that do not cater to those doctrines.
“Moreover, their cultural background gives them... there really is no precise term for it... the impulse to search for any means by which a godlike entity might be defeated, and to implement those means ruthlessly.”
The Second for Ecology looked up again. “Hubris?”
“Considered and rejected because of the connotations, but... it might be worth reconsideration in spite of that. Perhaps I shall try to make a technical term of it. Thank you.”
She smiled. “No charge.”
“As I was saying, to summarize: an alliance with the Commune against conventional opponents could prove problematic; an alliance against unusual or esoteric threats might be in order. That is, I think the essence of the matter. Was there anything else?”
The First Technarch shook his head. “No, I believe that was all.”
“Well then, ladies and gentlemen, by your leave I shall stop taking up your valuable time. Feel free to call upon my services at any time; it’s always a pleasure.”
“Goodbye, Geppetto. Thank you.”
There was a click on the audio system to indicate that the AI was no longer plugged in. Dr. O’Connell clapped his hands. “Well, I think his recommendations are solid. I propose that Cal, Max, Nancy, and I get together and start working on a set of ‘instructions’ for her to take with her to the Commune; the rest of our resolutions are fairly well established. Thank you all for your time and contributions, and unless there are any further remarks, I propose that this meeting be adjourned.”
There were none. But as the Technarchs filed to the door, Dr. Warren-Marshall turned to the Second for Research.
“Emloy... did it seem to you like Geppetto was controlling both sides of that conversation at once?”
There was a long silence.
“You know, now that I think about it, I suspect that he was. Subtly, but... yes. Perhaps it is just as well that he has always declined offers for a permanent seat...”
"
- PeZook
