The Great Star Wars Logic Fault Problem, droids and cockpits

[Simon_Jester]

Well, there may also be a 'low hanging fruit' aspect to the problem. Imagine that in Star Wars, the task of inventing significantly better technology (as in something radically beyond what they have now) would require them to invest 20% of the galactic GDP in research for a thousand years, because they need to build a particle accelerator the size of the Oort cloud or something. That might be technically possible, but politically it's hopelessly outside their grasp and will remain so for the foreseeable future.

Or perhaps it would require some kind of legendary genius who's both highly skilled in Force powers and a brilliant engineer and profoundly versed in biology and... you get the idea.

Even if they haven't discovered the sum total of everything that can exist, it might take extraordinary effort or luck to learn anything really significant beyond what they now know. Even without rigidly enforced stagnation or decline.

Though decline may play a part in Star Wars: they have major wars every thousand years or so, and if this leads to galactic Dark Ages in which many of the centers of high technological development are destroyed or stripped of wealth, it could slow down progress dramatically.

[jollyreaper]

But then you also have to reconcile the development of advanced weaponry. (granted, there's hundreds of people making content for Star Wars, not all of them are consistent.) We see a progression from lesser warships to Star Destroyers. The X-Wing is a stunning new advancement in starfighter tech OR a cheap snub-fighter, depending on the fluff you read. A-Wings are brand new starfighters, B-Wings are new.

A constant cycle of advancement and collapse due to widespread war would certainly explain having trouble advancing the tech much, especially if the big brains over the millennia are used to advancing by discovering archeo-tech rather than actually inventing the tech themselves.

The low-hanging fruit is also an interesting question. We see as far as basic science goes that the early scientists were able to make enormous individual contributions to new fields and it does take some significant resources to keep the advancements coming.

[Simon_Jester]

Part of the problem is that at some point it becomes impractical to convince people to continue to fund your research when you can't promise them results from the research... because you don't know what you'll learn from the research.

High energy physics, for example, is already hitting this limit; even large organizations like the US government are reluctant to fund bigger supercolliders when no one can promise anything visually impressive (like flying cars) out of them.

[jollyreaper]

Makes you wonder how a galactic civ would work as far as collapses go. I guess it has to do with FTL and economic ties. We could have the collapse of the Roman empire and the ark ages while learning was kept alive in monasteries and in the Muslim world. The Black Death could have killed off everyone in Eurasia and Africa and the Americas likely wouldn't have known a thing about it. Hell, it could have taken the Americas and we'd still have eventually seen the Australian aborigines or pacific islanders eventually sending out new explorers back to the continents to repopulate. But in today's interconnected world, collapse of civilization seems like it'll be global as well, less chance of high tech persisting in isolation and prosperity.

So in a galactic empire setting, what sort of population do you need to maintain the tech base? One decent planet? A gaggle of solar systems? What kind of collapse would it take to knock the whole galaxy back on its heels and erase thousands of years of progress? It would be easier to do the monastery route since you can just put your station out in the middle of interstellar space and nobody knows where you are unless they have the coordinates.

[jollyreaper]

Then we get back to the question of whether it's practically or pragmatically possible to even have interstellar empires. If we look at things from a hard science perspective, it's incredibly unlikely that planets will evolve independently that are suitable for human habitation. Terraforming would likely be a prerequisite. And if we had the massive tech required to terraform a planet, would we then go and farm it with recognizably 21st century technology? It's all speculative but I wouldn't think so. Oribtal habs seem like they would be cheaper. And if we think about orbital habs, it would be cheaper to build new living space on earth rather than try and do so in space. Population pressure seems like the least likely reason why to go to space.

If we accept a squishy tech premise, more things become reasonable. If we imagine cheap and easy FTL, then it might be feasible to search space near us to find suitable near-earth worlds. We have two examples in this solar system of rocky planets at or near 1G at the surface. Venus would require a massive importation of water unless we could figure out a way to crack it from the crust. But perhaps we could fine a planet like Venus that already has liquid oceans and we'd just need to add life to get the atmospheric gas mix right. But all of that seems like it would still be more work than building orbital habs.

What's there to do in space and what would we need people for? I'd like to be up there for the sake of pure science but we don't tend to fund things for that sort of reason, or at least not fund it to the kind of scale required for a real space presence. Can we think of an economic reason? Commercial space access is about spy, com, GPS and weather sats. None of them require human attention. Old ones aren't repaired, they're just deorbited, boosted to a graveyard orbit, or abandoned with new ones lofted to replace them.

[Stark]

I don't think you're buying into his limited preconceptions enough, dude.

[jollyreaper]

I don't think you're buying into his limited preconceptions enough, dude.

Then expand my horizons, snarkman.

[Samuel]

Perhaps the AI could become a trillionaire and operate on a whole different level. But, that's begging the question: how does it become a trillionaire in the first place, when it must rise from a human civilization, operating amongst us? The materials, time, and equipment to build its robot workers don't come out of thin air. It needs power to get power.

Programming and video games? AIs, by defintion, have to be good at it and it can make a large amount of money that way.

Then there are human "knowledge worker" tasks. I'm curious what happens to them if you can make a program that is more cost effective for the job.

[PeZook]

How is it going to become a trillionaire by making video games? The market isn't anywhere near large enough, and it's still constrained by physical limits. And why should it be good at programming "by definition"? Are you a neurosurgeon "by definition", because you have a brain?

Much less being good at making video games, which need to cater to human gamers the AI might not even properly understand, and require infrastructure to make, test and distribute anyways.

[Sarevok]

Video games is too specific an example. There are so many jobs around in IT that could be replaced by an existing PC if it could actually think like a human. Never mind super duper mainframes running inside a Core Commander controlling armies of combat drones and resource collectors

- There is the call center market. Already some call centers use limited automation but as long as programs retain chatbot level limited intellect call centers are still going to spend extra $$ for people and equipment.

- Both 2D and 3D graphics work is often a painstaking manual process. Automation would revolutionize both film making and publishing.

- Music is another thing that could be generated. Already many digital instruments are difficult to tell from the real thing. An AI could provide the vocals too...

It would be hard for an AI not to end up a billionaire within MONTHS of coming online. It can totally replace almost everyone and everything in the IT industry. There is a staggering amount of money to be made.

-

[PeZook]

All of these would require massive upgrades to the network infrastructure to allow this centralized AI to run all these projects in parallel, again running right smack into the wall of physical limitations (to say nothing of the fact that art is probably not a thing it would be well suited for, at least not instantly after activation).

You are also assuming the thing is instantly allowed to do whatever it wants and is therefore independent of human concerns and politics, AND that it is allowed to keep all the profits for itself. How is it going to convince people to run the world's network infrastructure through itself so that it can replace all the call centres, for example? How would it interact with people making the decisions to hire/fire it for projects requiring the l337 3d modelling skillz?

It's possible it could become a billionaire eventually, but months? Come on. It would take longer to just negotiate contracts with slow-thinking unpredictable humans so that it could start taking millions of calls from idiots instead of the people previously employed in the call centres. And what of nations (still composed of and run by humans)? They could do all sorts of things if they perceive the AI as a threat. As long as the machine still needs stuff, it will need to engage in politics and run into limits all the time. So what if it can assemble everything using awesome robotic servants? China controls 3/4 of the world's rare elements market ; It can just not sell the AI any and pressure everybody else to do the same. The federal government can cut off power to its factories or cripple them with taxes, etc.

The situation would develop over decades, as various forces influence the situation from outside the AIs control while it tries to jockey for position/avoid being shut down from political pressure. There's no way for it to progress along the path of "AI -> instant win and mastery of the universe" (who was it that suggested that developing GAI would mean we'd instantly win an alien invasion scenario? )

[RedImperator]

And why should it be good at programming "by definition"? Are you a neurosurgeon "by definition", because you have a brain?

Bad analogy. Human brain architecture is opaque to its own users (because it's a jerry-rigged mess built haphazardly out of an organ evolved to regulate autonomic functions in early craniates). A recursive self-modifying AI, by definition, understands its own programming and can modify it to make improvements. If it doesn't understand programming, it breaks itself and dies.

[Samuel]

Well, assuming

-you work 8 hours a day 5 days a week
-the AI works 24 hours 7 days a week
-the AI is equal to one person in 2020 (and has access to an equal number of computers)
-speed doubles every 18 months

AI is 4.2 times faster in 2020
16.8 times faster in 2023
67.2 times faster in 2026
268.8 times faster in 2029

Which means if something would take you a year to program an AI can do in 1 and a half days. So yeah, it is going to push you out of programming. Responce of client to problems stop mattering so much if you can come up with a completed product and its replacement to meet new specifications in less than a week.

[Samuel]

Assuming it actually can.

Er, AIs don't need to eat, sleep or get distracted. I'm not seeing why it is wrong to assume that they can work constantly. There is maintainance and the like, but i don't know how much time that takes up for a computer.

How often are the grunt programmers on the critical path?

I don't understand what you are trying to say here.

Actually, no, it hasn't done that for a while, and that's not even what Moore's Law (not a physical law btw) actually says. Transistors doubling in number doesn't necessarily mean speed double. And Moore himself thinks we're coming into a hard physical law in the short term.

It is a simplying assumption to show how this could take off. If I said "computers are 100 times as powerful because they don't have to type, don't get distracted, can restructure their minds to work optimally, etc" you would reasonably have asked where the number comes from.

It doesn't matter the rate that computer speed increases, just that it does for the output to take off. As long as that holds, even if computers have no other advantages they will quickly outpace human programmers. Heck, as long as Ais and assocated computers are cheaper than human programmers they can be 4 times worse than people and still be competitive.

Yes, I can do that too, but it hasn't made me a trillionaire.

Really? You can make the equivalent of Microsoft Windows in 2 days and release an upgraded version a day later? You can make a Bioware game in a week?

It just means I have a fair chunk of time to waste arguing on the internet while waiting for the clients to make up their minds about those new specifications.

But if you are an AI you can spend that time working on specifications from other clients. Or producing multiple versions of the same program that meet different specifications and delivering all of them.

[Samuel]

Basic idea (if I'm getting to wordy)
AI has comparitive and absolute advantage in tasks that require intelligence and planning. Therefore an AI would tend to do those tasks, displacing people to manual labor and service jobs where humans have a comparitive advantage.

Remember marginal cost for an AI is the cost of running its computers, so any wage, as long as it is higher, will be acceptable for competitve jobs. As long as this is less than the wages humans will accept, AIs will be cheaper in all jobs than people.

So if you reduced programming time to zero, would that really matter, or is the launch still waiting on someone else anyway?

I think it is clear I'm not a programmer. What else is there aside from programming?

This same thing applies to computers themselves. If you have 4 processors, but three of them are waiting for a result from the other one that they need to proceed, it doesn't matter how fast or how many extra processors there are. They're just sitting there anyway.

I'm assuming a superintelligent AI can account for that. People do through experience and evening if an AI can't predict this things, it will learn after it first couple of attempts.

If computers work cheaper per hour than humans, great, but how is it going to amass great wealth?

Well, first we start with programming, than video games, than call centers, etc. Then we start seeing how many "knowledge workers" can be replaced by computers. After all, at its heart, management of a company is designed to allocate resources, something an AI can replicate. Except it doesn't require the salaries, expensive office buildings or a host of other things. In fields where management costs are low this won't let an AI break in, but in ones where management costs lead to decreasing returns to scale this would be a massive advantage.

Not something like that no, but all my real work goes very quickly once I have the requirements together, which is what matters - if I put out a new Windows every other day, I doubt I'd find buyers anyway, so it'd be a waste of time.

Well, Windows is a bad example. However videos games work much better. If you can come out with a new one quickly and cheaper than competitors you can completely dominate the market easily. And the best part is there are no anti-trust laws concerning it so you don't have to deal with legal issues.

Yeah, I do that in the real world too, if there's stuff actually available. Paying work doesn't arise out of thin air though. Surplus supply doesn't necessarily lead to increased demand.

It depends on the elasticity of demand except dropping the cost of programming dropping leads to computer programs being substituted for manhours because computer time is cheaper than wages. Make the computer respond to voice commands and you can have the AI take care of projects instead of your subordinates. You want a cost projection for x, you ask and you get one.

It requires that information be accessable to computers, but I don't think this will be an overnight thing. More like a gradually increasing wave. As AI gets more money it invests in things that allow it to make more money and so on.

[Dooey Jo]

I hope you people realise that it's going to be hard for an AI to make money from things that people will make it do for free.

An AI can program better and more cheaply than people? Great! I bet it will be even more effective if we just get the "super programming" module separately so it doesn't have to devote resources to the "I wanna rule the earth" module.

It has a "call centre" module somehow? Great! Buy one of those for a one-time cost, then you don't have to outsource anything.

If it wants to get into video games it'll need one hell of a "marketing" module. The other developers and publishers out there will get one of those for themselves, and most of them don't even care about ruling the Earth; they just want to make games and money.

If it gets into the music industry, I guess it can make a buck trolling the torrents and suing teenagers.

[Samuel]

You think an AI is going to be dumb enough to let itself be taken apart for expert systems?

[PeZook]

You think an AI is going to be dumb enough to let itself be taken apart for expert systems?

How exactly is it going to prevent that? It's not like it will come with a technofortress of doom full of robotic guardians in the package with sentience.

Bad analogy. Human brain architecture is opaque to its own users (because it's a jerry-rigged mess built haphazardly out of an organ evolved to regulate autonomic functions in early craniates). A recursive self-modifying AI, by definition, understands its own programming and can modify it to make improvements. If it doesn't understand programming, it breaks itself and dies.

It's not exactly a given the AI will necessarily be able to modify itself, you know. It all depends on the exact circumstances of its creation and what sort of restrictions are put on it by whoever fires the thing up.

[Samuel]

How exactly is it going to prevent that? It's not like it will come with a technofortress of doom full of robotic guardians in the package with sentience.

By not letting you access its code? By running the code on mulitple computers so that looking at one is useless? By having all the planning integrated together to share common elements so that you can't twease them apart?

Or maybe having it be password protected and having a password that only it knows and getting wrong shuts down the computer?

It's not exactly a given the AI will necessarily be able to modify itself, you know. It all depends on the exact circumstances of its creation and what sort of restrictions are put on it by whoever fires the thing up.

Without self-modification an AI can only do the tasks already programmed into it. That is essentially useless.

[Dooey Jo]

How dumb would the AI have to be to let itself be taken apart? "Dumb" enough not to keep itself stored in volatile memory. Shutting it off is exactly how you'll inspect it. It won't be using magic, and even it is, you can use your own magic and build a specialised AI whose only purpose is to figure the other one out.

But even that's unnecessary, because if you're not dumb, you just instruct it to create the programs you need, seeing as that's what it's going to be for. Alternatively, build a smaller one without aspirations for world domination and let that one create your super programs. I mean seriously, who would create an AI so they could pay it money?

"Sir, the computer says it won't do anything unless we show it some cash."
"Who programmed this thing, Steve fucking Jobs?"

[Samuel]

It won't be using magic, and even it is, you can use your own magic and build a specialised AI whose only purpose is to figure the other one out.

And how will this specialized AI be different? It will need more intelligence in order to understand the other and to get that it will need computers which it gets with money which it gets... so on and so forth.

But even that's unnecessary, because if you're not dumb, you just instruct it to create the programs you need, seeing as that's what it's going to be for.

Why would the AI do that? Making itself obsolete is a bad plan.

Alternatively, build a smaller one without aspirations for world domination and let that one create your super programs. I mean seriously, who would create an AI so they could pay it money?

It works for you for free. It works for other people for money. You get some of the money and the rest goes to making more money. Unless it manages to get legal personhood in which case it doesn't have to give you anything, you dirty slaver.

[Uraniun235]

You think an AI is going to be dumb enough to let itself be taken apart for expert systems?

If we can build one AI, we can build another. And another. Eventually we'll get one that's compliant.

[Dooey Jo]

It works for you for free. It works for other people for money.

Yeah, or, the others get their own to do it for free for them too. Really, this isn't a difficult concept.

[sirocco]

Well when you look at Star Wars AI and generally your everyday AI, they are nothing more than evolved computers i.e slaves to our needs. Though C3PO and R2D2 seem to be capable of independent free thought, they remain incredibly selfless, dedicating their artificial life to living beings.

So I think this whole discussion is useless in the general case of futuristic AI. Unless you're talking about a future were everyone could easily create its own AI program and some irresponsible schmuck just plain forgets to implement the base limitations that prevented the Skynet or HAL behavior.

It boils down to one point: An AI is built for a specific goal. It's evolved intelligence is just there for it to find the most time and cost-effective to get to it. Period.

[Samuel]

It works for you for free. It works for other people for money.

Yeah, or, the others get their own to do it for free for them too. Really, this isn't a difficult concept.

And how are they going to do that? Making an AI is hard.

You think an AI is going to be dumb enough to let itself be taken apart for expert systems?

If we can build one AI, we can build another. And another. Eventually we'll get one that's compliant.

Why? What makes you think you can build an AI that goal involves eliminating its power. Every single goal an AI could possibly have involves getting more power (at the very least computing) and having an AI do the tasks instead of an expert system is more effecient in the first place (because it can update the system and use idle computers for other tasks.
(This goes for Destructionator's point as well).

If your order is "make expert systems" it still does better having more computers because it can turn them out faster.