MegaComp vs MegaAliens... (think i wrote myself into corner)

SF: discuss futuristic sci-fi series, ideas, and crossovers.

Moderator: NecronLord

Post Reply
User avatar
Crossroads Inc.
Emperor's Hand
Posts: 9233
Joined: 2005-03-20 06:26pm
Location: Defending Sparkeling Bishonen
Contact:

MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by Crossroads Inc. »

First off let me say its not really one group "vs" another, but more of two groups that are going to butt heads in an unpleasant manor. Let me lay things out...

Long long ago, as I was weaving my own little universe, of the many various factions I slowly created, one was "RUDI" a super intelligent mega computer that originally, was just in charge of a big honking ship, and not much else.. The second where the 'Skothians' who started out as a sort of mysterious mystical group of aliens.

Over time as my story lines and plot has become more and more complex, both have evolved considerably.

The 'Skothians' grew into a sort of generic 'The Ancients' race, a race that was massively more advanced then any other in the Sector of space where all my stories take place. They entered the sector hundreds of years ago, noticed several sapient races in the midst of advancing toward SPace, and, being immensely powerful, natural meddlers, and quite frankly 'bored' played a sort of 'real life Galactic Civ' game with the races. The helped unify some factions ot make them able to withstand others that they knew would eventually attack, they would give tech, but only just enough ot make fights and wars on par, and thus 'more fun' to watch.
Most in the Sector have no clue how callus or how depe their meddling is and see hold them in reverence and awe.

Now for the second part 'RUDI'

He stared as just another supercomputer, because All scifi lovers want a super intelligent computer... His role in things grew from being in charge of just a ship, to 'accident;y' coming into being after being installed in a newly built central military Space Fortress... Mostly he has changed as I have gone out of my way to Buck the trends of other SciFi computers... I Have given him a sense of humor, advanced emotions, a sense of style, art, music, Etc. His 'Origin story' as it stands now stems from the melding of two large mainframes ((with some 'ancient' tech mixed in)) that had slowly absorbed the personalities and mental patterns of the countless engineers who worked on it.. Over a period of years he grew from being somewhat self aware to a fully fledge personal of his own.

He also has 'read up' on his own stereotypes as it where.. The other races liek ours is full of fiction involving computers going mad and taking over trying to destroy biological life. Learning from this as it where, he has gone a more personal rote.. He has formed businesses, has PR people, became immensely rich producing computer systems no one else could dream of matching, video games, entertainment, work and labor robots. He has highly lifelike robots he uses as Avatars to do everything form appear at press conferences to doing Sunday Morning talkshows. At the current point he is like a celebrity that others have almost forgot at this point is a machine that currently has 'clones' of himself in virtually all military and civilian hardware...

WHERE is all this leading?

Well in case you havn't spotted it, these two groups are invariably going to clash...
The Skothians, given their nature to meddle and control the lesser races, would quickly realize how much of a 'threat' RUDI would become, it wouldn't take them long to realize all his wonderful products have various backdoors and dormant clones installed, meaning he could tak control of military resources wiht a simple command.
RUDI for his part, able to acces virtually any computer in the Sector (minus the Skothians) would not take long to put the pieces together and realize just how much they have been doing behind the scenes. RUDI may be on the cusp of all out, though peaceful, control of the races in the Sector, but its mostly because he sees them all as 'his' Like Durendal, he wants nothing more then preservation of his parent races and plans for them to do anything to ensure their longevity... KNowing what the Skoothians have done, and could do, he would inverably seek to out them.

SO I find myself stuck, I have two groups that by thier natural would never let the other to get to the point they are now, and am wondering how I might change things,,,

Any Help?
Praying is another way of doing nothing helpful
"Congratulations, you get a cookie. You almost got a fundamental English word correct." Pick
"Outlaw star has spaceships that punch eachother" Joviwan
Read "Tales From The Crossroads"!
Read "One Wrong Turn"!
User avatar
White Haven
Sith Acolyte
Posts: 6360
Joined: 2004-05-17 03:14pm
Location: The North Remembers, When It Can Be Bothered

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by White Haven »

Could always go the Mexican Standoff route. To extend the Galactic Civilizations metaphor, RUDI tells the bug-eyed monsters, "Back off a bit, or your secret's out and you guys get to play the part of the Dreadlords in the sequel. You might win, you might lose, but either you all die, or you end up deprived of entertainment."

That's not to say that would work, but it's one thought that comes to mind. Alternately, of course, you could turn it into grand space opera and have RUDI start upgunning the various star nations while either fighting a holding action to buy time or baffling your local weakly godlike entities with bullshit for much the same reason.

Alternately alternately, have RUDI fake his own death in an 'anti-AI pogrom' and go silent-running, which could then tie into upgunning much more quietly if you felt like it.
Image
Image
Chronological Incontinence: Time warps around the poster. The thread topic winks out of existence and reappears in 1d10 posts.

Out of Context Theatre, this week starring Darth Nostril.
-'If you really want to fuck with these idiots tell them that there is a vaccine for chemtrails.'

Fiction!: The Final War (Bolo/Lovecraft) (Ch 7 9/15/11), Living (D&D, Complete)Image
User avatar
JGregory32
Padawan Learner
Posts: 286
Joined: 2007-01-02 07:35pm
Location: SFU, BC, Canada

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by JGregory32 »

You could go with RUDI actually being helped along by a Skothian who wants to up the difficulty of the game a little be introducing a new player.

Though you could always go with the RUDI being helped by an old race that got used and discarded by the Skotians out for revenge by building the only thing that can challenge hem.
Image
Be the Ultimate Ninja! Play Billy Vs. SNAKEMAN today!

Ian Malcolm: God creates dinosaurs. God destroys dinosaurs. God creates man. Man destroys God. Man creates dinosaurs.
Ellie Sattler: Dinosaurs eat man … woman inherits the earth.
Jurassic Park
User avatar
White Haven
Sith Acolyte
Posts: 6360
Joined: 2004-05-17 03:14pm
Location: The North Remembers, When It Can Be Bothered

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by White Haven »

*sniggers* Now I've got the mental image of a squabble between Skothians.

<Skothian helping RUDI>: This shit's too easy, here, let's put it on Heroic mode.

<Conventional Skothians>: Dammit, cut that shit out!

<SHR>: Fucking casuals...
Image
Image
Chronological Incontinence: Time warps around the poster. The thread topic winks out of existence and reappears in 1d10 posts.

Out of Context Theatre, this week starring Darth Nostril.
-'If you really want to fuck with these idiots tell them that there is a vaccine for chemtrails.'

Fiction!: The Final War (Bolo/Lovecraft) (Ch 7 9/15/11), Living (D&D, Complete)Image
Junghalli
Sith Acolyte
Posts: 5001
Joined: 2004-12-21 10:06pm
Location: Berkeley, California (USA)

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by Junghalli »

As I understand, the problem is with the Fridge Logic issues that the Skothians would have to be stupid to allow something like RUDI to arise, and once RUDI detects the Skothians he should blow their cover.

Problem # 2 is fairly easy. If everybody thinks the Skothians are benevolent they must have invented reasonably plausible lies to make themselves look benevolent. RUDI being superintelligent is not a guarentee of him being able to see through their lies if he's working with limited information. He may have no reason to suspect their true nature, or he may suspect it but not have any firm proof.

For Problem # 1 I can think of a couple of possibilities.

1) The Skothians are transaliens, or they have superintelligent AIs of their own. RUDI isn't a serious threat to them. He's superintelligent and is gearing up for an AI Singularity, big deal, the Skothians have already gone there long ago and they have a huge advantage in starting military hardware, industrial capacity, and intelligence over him. This would fit with White Haven's suggestion that the Skothians do not want to get rid of RUDI or he has a Skothian patron.

2) They were unaware of RUDI's creation and by the time they found out about him he was too powerful for them to take him out easily.

3) Also of course there's White Haven's suggestion: RUDI faked his own demise and is working behind the scenes.

4) RUDI's patron is a dissident Skothian who disaproves of the way the Skothians fuck with "lesser" races for the lulz and wants to help the other races fight back.

Also, one humble suggestion. If you wanted to subvert Hollywood cliches about evil robots, I think it would be even cooler to have a benevolent computer that you root for as one of the heroes that's driven entirely by pure cold logic. I see where you're coming from in wanting to get away from cliches but I personally find something more than a little human-chauvinist in the idea that a benevolent computer should have emotions. It smells of the same chauvinistic assumptions that go into Pinnochio Syndrome (being made of meat is the most awesome thing ever so of course an AI would want to be like us, our cognitive quirks are totally awesome and any mind that didn't have them and wasn't pure evil would desire them, a person that doesn't feel emotions is in some sense not complete). You could still keep the personable outer presentation of the AI - it would logically want to make people feel at ease in its presence, and I imagine humans might find something that would act as if it had human-like emotions more comfortable than something that would act completely emotionlessly. You could even keep the art appreciation and stuff - it would want to be able to talk to humans about their interests. That said I'll admit I'm reading my own pet peeves into your work there so I may be misjudging your work horribly, and if the origin story makes it sound like RUDI is some sort of hive-mind formed of human uploads in which case a high degree of biomorphism would make sense. I hope that was constructive.
Lord of the Abyss
Village Idiot
Posts: 4046
Joined: 2005-06-15 12:21am
Location: The Abyss

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by Lord of the Abyss »

OK, here's my idea. Postulate that RUDI level AIs are nearly impossible to build in-universe because the organic-level intelligences just aren't smart enough to properly build something with a higher order intelligence. Many have tried ( including the Skothians ), but the AI either crashes, or goes psychotic, flails around doing damage then crashes. So the Skothians know that RUDI is being built, but don't interfere expecting either a simple failure or an entertaining disaster; preoccupied by more important matters they aren't even playing close attention.

But by sheer luck ( and being too ignorant to know that it "doesn't work" ) RUDI's builders get it right and RUDI works properly. And for some time after it being built the Skothians don't actually realize it HAS worked right; it's such an ingrained assumption for the Skothians that AI doesn't work that they assume that the work isn't complete or that RUDI's builders are covering up their failure. By the time they realize RUDI is that once-in-millenniums achievement, a working high order AI, RUDI has begun establishing itself and is sniffing out their interference.
"There are two novels that can change a bookish fourteen-year old's life: The Lord of the Rings and Atlas Shrugged. One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs." - John Rogers
User avatar
Darth Hoth
Jedi Council Member
Posts: 2319
Joined: 2008-02-15 09:36am

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by Darth Hoth »

Junghalli wrote:Also, one humble suggestion. If you wanted to subvert Hollywood cliches about evil robots, I think it would be even cooler to have a benevolent computer that you root for as one of the heroes that's driven entirely by pure cold logic.
The content of this sentence is paradoxical. Someone who has no emotions will not appear sympathetic to humans, no matter what you intend as a writer; by our definition such an entity is the perfect sociopath. Even a benevolent entity driven by "pure logic" will inevitably be perceived as a villain, given that his merciless utilitarianism will be utterly opposed to human sensibilities.

Such creatures do, however, tend to make awesome villains, although that, too, may be cliché by now.
"But there's no story past Episode VI, there's just no story. It's a certain story about Anakin Skywalker and once Anakin Skywalker dies, that's kind of the end of the story. There is no story about Luke Skywalker, I mean apart from the books."

-George "Evil" Lucas
Junghalli
Sith Acolyte
Posts: 5001
Joined: 2004-12-21 10:06pm
Location: Berkeley, California (USA)

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by Junghalli »

Lord of the Abyss wrote:OK, here's my idea. Postulate that RUDI level AIs are nearly impossible to build in-universe because the organic-level intelligences just aren't smart enough to properly build something with a higher order intelligence. Many have tried ( including the Skothians ), but the AI either crashes, or goes psychotic, flails around doing damage then crashes. So the Skothians know that RUDI is being built, but don't interfere expecting either a simple failure or an entertaining disaster; preoccupied by more important matters they aren't even playing close attention.

But by sheer luck ( and being too ignorant to know that it "doesn't work" ) RUDI's builders get it right and RUDI works properly. And for some time after it being built the Skothians don't actually realize it HAS worked right; it's such an ingrained assumption for the Skothians that AI doesn't work that they assume that the work isn't complete or that RUDI's builders are covering up their failure. By the time they realize RUDI is that once-in-millenniums achievement, a working high order AI, RUDI has begun establishing itself and is sniffing out their interference.
I remember Starglider talking about the difficulties of arranging an AI's goal system to be Friendly. It could be that while creating AI is relatively easy making it not want to kill you is very difficult. Although then the Skothians would be worried about having to deal with the postsingularity homocidal AI civilizations, which would probably be even worse than dealing with pissed off transhumans or pissed off organics with servile AIs. At least evolution would tend to give organic creatures a sense of empathy which might translate into a reluctance to completely exterminate an enemy.

Maybe the Skothians manipulate things to deliberately push AI research into strategies that are extremely unlikely to succeed.
Darth Hoth wrote:The content of this sentence is paradoxical. Someone who has no emotions will not appear sympathetic to humans, no matter what you intend as a writer; by our definition such an entity is the perfect sociopath. Even a benevolent entity driven by "pure logic" will inevitably be perceived as a villain, given that his merciless utilitarianism will be utterly opposed to human sensibilities.
The first part may have some merit, but it's questionable whether an emotional superintelligent AI would be easy for humans to relate to either. The second part ... leads to a swamp because it begs the question of how exactly one would go about translating "benevolence" into a goal structure in a rational AI. For certain values of benevolence you could indeed program an AI to reflect values which's enforcement that would piss off a minimal number of humans. The simplest way to do this, conceptually anyway (I'm not sure about the implementation side), might be to have the AI attempt to model a consensus of what the human race as a whole would want it to do if they could vote on it and go with that, with limiters established on some combination of utilitarianism and rights based ethics. Personally I'd consider such an approach a mistake, but it would nicely approximate the way human societies make laws and so avert the dramatic trap of having the AI act in ways that would look like Knight Templar behavior to a lot of modern readers, which an AI programmed with some variant of utilitarianism alone probably would do. I would go this far though. The situation is one in which humans and other races are being victimized by a bunch of dicks who propogate vast suffering purely for their personal amusement. The computer's goal is to end this, with a minimum of blood being shed, and to then safeguard the continued existence of humanity and other sapience and perhaps to dramatically raise our standard of living. Even if it wasn't a naturally empathetic character I'm really not getting "villain" from that.
User avatar
Crossroads Inc.
Emperor's Hand
Posts: 9233
Joined: 2005-03-20 06:26pm
Location: Defending Sparkeling Bishonen
Contact:

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by Crossroads Inc. »

Well Hell, I ask for advice and I got, quite a bit in here to think about and quite a few pretty good ideas!

I think one of the big things all of this feed back has made me think about is that:
RUDI is probably less likely to oust/expose the Skothians, as the Skothians are to be worried about his coming into existence.
Darth Hoth wrote:
Junghalli wrote:Also, one humble suggestion. If you wanted to subvert Hollywood cliches about evil robots, I think it would be even cooler to have a benevolent computer that you root for as one of the heroes that's driven entirely by pure cold logic.
The content of this sentence is paradoxical. Someone who has no emotions will not appear sympathetic to humans, no matter what you intend as a writer; by our definition such an entity is the perfect sociopath. Even a benevolent entity driven by "pure logic" will inevitably be perceived as a villain, given that his merciless utilitarianism will be utterly opposed to human sensibilities.

Such creatures do, however, tend to make awesome villains, although that, too, may be cliché by now.
This is the crux of the issue behind how I had been perceiving RUDI recently. In truth the idea of a Supercomputer intelligence that could laugh, tell jokes, and was perceived as a jovial personality by others, was something I thought would be 'cool' years and years ago, long before I realized how important it would be for him. As Darth Hoth alludes to, one of the reason why virtually every, cold, logical, emotionless computer in Fiction turn out to be heartless bastards and villains is they have No Emotion, no sense of pity, compassion, sympathy, empathy, etc. Shoot even VULCANS have emotions, they just suppress them.

Junghalli wrote: The first part may have some merit, but it's questionable whether an emotional superintelligent AI would be easy for humans to relate to either. The second part ... leads to a swamp because it begs the question of how exactly one would go about translating "benevolence" into a goal structure in a rational AI. For certain values of benevolence you could indeed program an AI to reflect values which's enforcement that would piss off a minimal number of humans. The simplest way to do this, conceptually anyway (I'm not sure about the implementation side), might be to have the AI attempt to model a consensus of what the human race as a whole would want it to do if they could vote on it and go with that, with limiters established on some combination of utilitarianism and rights based ethics. Personally I'd consider such an approach a mistake, but it would nicely approximate the way human societies make laws and so avert the dramatic trap of having the AI act in ways that would look like Knight Templar behavior to a lot of modern readers, which an AI programmed with some variant of utilitarianism alone probably would do. I would go this far though. The situation is one in which humans and other races are being victimized by a bunch of dicks who propogate vast suffering purely for their personal amusement. The computer's goal is to end this, with a minimum of blood being shed, and to then safeguard the continued existence of humanity and other sapience and perhaps to dramatically raise our standard of living. Even if it wasn't a naturally empathetic character I'm really not getting "villain" from that.
To discuss the whole being able to relate to a Supercomputer... Well the way I have constructed things, RUDI stops 'being' a supercomputer, as far as most see him. Imagine for a few years, the planets hear about some new Supercomputer that seems intelligent, well, thats a little worrying, how odd... But its on some space station in the middle of now where, nothing to worry about. Years later people might have heard about some big deal about some new lifeform.. Or maybe they heard about small computer businesses and factories being bought up by some unknown investor. Maybe they hear some of that.. But what they REALLY know is that this 'Robot' has shown up and seems very nice and likable! There is an Android that says he likes people and just wants to get to know everyone, and what is the harm in that?

One of the conventions of Fictional AI's that usually lead to the whole taking over things is the whole "If sapients are left in charge, you shall eventually destroy yourselves! Only me, a giant computer, can save you from yourselves!!!'" Well, by RUDI's time, most of the worlds in the area have passed this danger, they have off world colonies, they are relatively unified and have passed the phase they might blow themselves up, and worry more about someone from another planet doing it. As such RUDI isn't worried about controlling the lives of people, they seem to be doing an ok job of it. His long term plan is still to ensure their survival and continued existence as a Group of races, and the best way for that at this point is the integration of himself into the Allied Worlds military.

And in that vein comes the compromise that I've been working on for this whole deal... Thanks to a few ideas already posted as well as some additional stuff I think I have a satisfactory solution that mostly revolves a much more gradual 'evolution' of RUDI From big computer into Sapient emotional super business owner. At the point when he and the Skothians would butt heads, I've considered: RUDI's goal is the long term continuation of the various Sapients that led to his coming into existence. While he won't exactly be pleased that the Wars the Sector have gone through have been looked upon as Entertainment for the SKothians, He knows enough that they arn't out to Cause wars or incite conflict between the other races. More to the point the amount of Tech they have would go a very long way to ensuring his races live a very long time.

On the flip side, once the SKothians realize that the AI That, at first glance didn't seem that much more special then their own, semi-aware AI's is now suddenly producing most electrical goods for the Sector planets, I am betting their innate desire to control and manipulate things will have their leaders seeking RUDI out to perhaps get him 'on bored' with their agendas, weather THAT Works I am not sure yet...
Praying is another way of doing nothing helpful
"Congratulations, you get a cookie. You almost got a fundamental English word correct." Pick
"Outlaw star has spaceships that punch eachother" Joviwan
Read "Tales From The Crossroads"!
Read "One Wrong Turn"!
User avatar
White Haven
Sith Acolyte
Posts: 6360
Joined: 2004-05-17 03:14pm
Location: The North Remembers, When It Can Be Bothered

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by White Haven »

The are, however, apparently out to make you capitalize random words for no discernible reason.
Image
Image
Chronological Incontinence: Time warps around the poster. The thread topic winks out of existence and reappears in 1d10 posts.

Out of Context Theatre, this week starring Darth Nostril.
-'If you really want to fuck with these idiots tell them that there is a vaccine for chemtrails.'

Fiction!: The Final War (Bolo/Lovecraft) (Ch 7 9/15/11), Living (D&D, Complete)Image
User avatar
Crossroads Inc.
Emperor's Hand
Posts: 9233
Joined: 2005-03-20 06:26pm
Location: Defending Sparkeling Bishonen
Contact:

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by Crossroads Inc. »

White Haven wrote:The are, however, apparently out to make you capitalize random words for no discernible reason.
I have had teachers noting that in papers since I was in 3rd grade. Ive tried everything to keep it in check, usually doing short responses its never an issue. But for whatever reason when I begin to write or type big responses I start randomly capitalizing words. :P
Praying is another way of doing nothing helpful
"Congratulations, you get a cookie. You almost got a fundamental English word correct." Pick
"Outlaw star has spaceships that punch eachother" Joviwan
Read "Tales From The Crossroads"!
Read "One Wrong Turn"!
User avatar
Terralthra
Requiescat in Pace
Posts: 4741
Joined: 2007-10-05 09:55pm
Location: San Francisco, California, United States

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by Terralthra »

Darth Hoth wrote:
Junghalli wrote:Also, one humble suggestion. If you wanted to subvert Hollywood cliches about evil robots, I think it would be even cooler to have a benevolent computer that you root for as one of the heroes that's driven entirely by pure cold logic.
The content of this sentence is paradoxical. Someone who has no emotions will not appear sympathetic to humans, no matter what you intend as a writer; by our definition such an entity is the perfect sociopath. Even a benevolent entity driven by "pure logic" will inevitably be perceived as a villain, given that his merciless utilitarianism will be utterly opposed to human sensibilities.

Such creatures do, however, tend to make awesome villains, although that, too, may be cliché by now.
That depends entirely on the goal system. You have pre-assumed a goal system based on some caricature of utilitarianism and characterized it as "merciless." Until and unless you define the supergoal system, assuming any sort of outcome - unsympathetic, sociopathic, merciless - is pretty facile.
Junghalli
Sith Acolyte
Posts: 5001
Joined: 2004-12-21 10:06pm
Location: Berkeley, California (USA)

Re: MegaComp vs MegaAliens... (think i wrote myself into corner)

Post by Junghalli »

Crossroads Inc. wrote:This is the crux of the issue behind how I had been perceiving RUDI recently. In truth the idea of a Supercomputer intelligence that could laugh, tell jokes, and was perceived as a jovial personality by others, was something I thought would be 'cool' years and years ago, long before I realized how important it would be for him.
An emotionless superintelligent AI could still laugh, tell jokes, and be percieved as a jovial personality by humans. If it wants humans to be comfortable around it adopting a persona humans would find likable would be a good idea. The idea that you need to have human emotions in order to convincingly act like a human is a fallacy. All you need is a good understanding of how the human mind works, and good acting ability. A superintelligence should be able to easily acquire both. Actually even the first requirement is questionable; I suspect if you were intelligent enough you could probably fake humanity pretty effectively just by observing humans and imitating them, even if you had a very imperfect understanding of how our minds actually worked (although obviously with this approach you would be vulnerable to slip-ups).
As Darth Hoth alludes to, one of the reason why virtually every, cold, logical, emotionless computer in Fiction turn out to be heartless bastards and villains is they have No Emotion, no sense of pity, compassion, sympathy, empathy, etc. Shoot even VULCANS have emotions, they just suppress them.
The idea that those things are required for benevolence is another fallacy. Benevolence is a matter of goals. Emotions are simply one method of creating and enforcing a goal system, and there's no reason to believe they're the ideal method. In fact I'd say there are good reasons to think that human-like emotions, at any rate, are far from an ideal goal-enforcement method, as in humans you can observe them creating all sorts of faulty results like people feeling pity, sympathy, and compassion for things that they know they aren't actually conscious (the entire concept of drama in fiction is only possible because of this cognitive glitch).
One of the conventions of Fictional AI's that usually lead to the whole taking over things is the whole "If sapients are left in charge, you shall eventually destroy yourselves! Only me, a giant computer, can save you from yourselves!!!'"
Also to be considered is the fact that, quite frankly, a genuinely benevolent superintelligence would almost certainly do a much better job running the world than we are doing now. A lot of the involuntary suffering that happens today is either created or exacerbated by acts of humans. It isn't just the threat of humans blowing themselves up that could cause a benevolent superintelligence to take one look at our world and decide it needs to establish control for our own good; it's stuff like people dying of preventable diseases and starvation by the millions when this probably could be prevented if we really tried but isn't because human civilization has a massive institutional commitment to the present shitty system. Similar logic could apply here, of course; one would hope that a humanity that can travel to the stars would have solved many of the problems of our present world.
Post Reply