Computer derives natural laws from raw data

[Natorgator]

I thought this was really neat. From Physorg:

If Isaac Newton had access to a supercomputer, he'd have had it watch apples fall - and let it figure out the physical matters. But the computer would have needed to run an algorithm, just developed by Cornell researchers, which can derive natural laws from observed data.

The researchers have taught a computer to find regularities in the natural world that become established laws - yet without any prior scientific knowledge on the part of the computer. They have tested their method, or algorithm, on simple mechanical systems and believe it could be applied to more complex systems ranging from biology to cosmology and be useful in analyzing the mountains of data generated by modern experiments that use electronic data collection.

The research will be published in the journal Science (April 3, 2009) by Hod Lipson, Cornell associate professor of mechanical and aerospace engineering, and graduate student Michael Schmidt, a specialist in computational biology.

Their process begins by taking the derivatives of every variable observed with respect to every other - a mathematical way of measuring how one quantity changes as another changes. Then the computer creates equations at random using various constants and variables from the data. It tests these against the known derivatives, keeps the equations that come closest to predicting correctly, modifies them at random and tests again, repeating until it literally evolves a set of equations that accurately describe the behavior of the real system.

Technically, the computer does not output equations, but finds "invariants" - mathematical expressions that remain true all the time.

"Even though it looks like it's changing erratically, there is always something deeper there that is always constant," Lipson explained. "That's the hint to the underlying physics. You want something that doesn't change, but the relationship between the variables in it changes in a way that's similar to [what we see in] the real system."

Once the invariants are found, potentially all equations describing the system are available: "All equations regarding a system must fit into and satisfy the invariants," Schmidt said. "But of course we still need a human interpreter to take this step."

The researchers tested the method with apparatus used in freshman physics courses: a spring-loaded linear oscillator, a single pendulum and a double pendulum. Given data on position and velocity over time, the computer found energy laws, and for the pendulum, the law of conservation of momentum. Given acceleration, it produced Newton's second law of motion.

The researchers point out that the computer evolves these laws without any prior knowledge of physics, kinematics or geometry. But evolution takes time. On a parallel computer with 32 processors, simple linear motion could be analyzed in a few minutes, but the complex double pendulum required 30 to 40 hours of computation. The researchers found that seeding the complex pendulum problem with terms from equations for the simple pendulum cut processing time to seven or eight hours.

This "bootstrapping," they said, is similar to the way human scientists build on previous work.

Computers will not make scientists obsolete, the researchers conclude. Rather, they said, the computer can take over the grunt work, helping scientists focus quickly on the interesting phenomena and interpret their meaning.

[Starglider]

Demonstration programs that performed similar feats actually existed in early 1980s, but they weren't really practical due to either a) being based on binary logic and thus restricted to very simply expressible scenarios or b) having computational requirements that rendered anything but the most trivial problems impractical. As with AI in general, having (relatively) huge amounts of computing power available with modern equipment makes some things a lot easier, but ultimately doesn't help much on the really hard problems (i.e. generalising at higher levels of abstraction).

[Singular Intellect]

Demonstration programs that performed similar feats actually existed in early 1980s, but they weren't really practical due to either a) being based on binary logic and thus restricted to very simply expressible scenarios or b) having computational requirements that rendered anything but the most trivial problems impractical. As with AI in general, having (relatively) huge amounts of computing power available with modern equipment makes some things a lot easier, but ultimately doesn't help much on the really hard problems (i.e. generalising at higher levels of abstraction).

Damnit StarGlider, so you're saying this is not a critical step towards the AI Singularity? I'm getting impatient here!

[Starglider]

Damnit StarGlider, so you're saying this is not a critical step towards the AI Singularity? I'm getting impatient here!

It's interesting on a technical level, but it's essentially a combination of data mining techniques that have been common for the last decade or so with an application area that was first attempted thirty years ago, so it would be incorrect to consider it a quantum leap in capability. I will say that our own low-key general AI research program is targetting something comparable as a medium term goal. However our goal is to get the automated programming system to design and employ customised machine learning algorithms for a problem area (e.g. articulated system motion analysis, specific vision problems etc), i.e. doing what these researchers did automatically. This recursion problem is, as Kuroneko might say, 'non trivial' - currently the system works ok on (selected) toy problems, but not anything realistic.

[Kwizard]

Then the computer creates equations at random using various constants and variables from the data. It tests these against the known derivatives, keeps the equations that come closest to predicting correctly, modifies them at random and tests again, repeating until it literally evolves a set of equations that accurately describe the behavior of the real system.

I'm sure the inner workings of this algorithm are interesting in their own way, but it's rather... unimpressive for a purportedly "intelligent" agent to be using the same kind of strategy natural selection applies to organisms when it's maximizing genetic fitness. At least as far as I'm aware, it's not like actual physicists arrive at the correct models through burning up obscene amounts of time and other resources re-arranging equations at random.

[Singular Intellect]

At least as far as I'm aware, it's not like actual physicists arrive at the correct models through burning up obscene amounts of time and other resources re-arranging equations at random.

Exactly. Right now computers are simply impressive because they can do this in such short time frames (relatively).

If they were given actual intelligence with that kind of processing power...it's almost worthy of worship in my eyes.

[Starglider]

I'm sure the inner workings of this algorithm are interesting in their own way, but it's rather... unimpressive for a purportedly "intelligent" agent to be using the same kind of strategy natural selection applies to organisms when it's maximizing genetic fitness. At least as far as I'm aware, it's not like actual physicists arrive at the correct models through burning up obscene amounts of time and other resources re-arranging equations at random.

Actually quite a few AI academics have argued that the human brain uses evolutionary processes both to learn (see Edelman's 'Neural Darwinism' movement) and to make decisions at a subconscious level (e.g. W.H. Calvin's theories, but a lot of other people have different versions of the same thing). Honestly we don't know enough about higher brain function to say either way at present, but it seems to me that both of these theories are plausible, but probably in a less comprehensive way than their proponents claim (unsurprising, almost all theories of brain function are oversimplified by necessity). The massive parallelism of the brain does in fact give it 'obscene' computational resources, but these cannot be directly applied directly to something as symbolic and abstract as written equations, so exactly how intuition contributes to scientific discovery is a hotly contested debate.

As for the software, bear in mind that most genetic algorithms are in fact rather more complex than 'parameters are picked out of a hat'. There may be quite a few different mutation/recombination operators all with their own probability distributions over the actual operation performed, which in advanced systems may be tweaked on the fly, plus there are global issues such as culling method/harshness and population size, count and migration mechanisms. Furthermore in practical data-mining applications GAs are often chained with pre- or post-processing steps that employ other machine learning algorithms (e.g. SVMs and GAs are often complimentary in applications where a GA can discover complex structural features, but the SVM is much better at local optimisation). This doesn't come across in popsci-level articles.

Typically I find myself arguing that (purely) genetic algorithms are inefficient in the general case and genetic programming is both inefficient and dangerous, but that doesn't change the fact that with the current state-of-the-art, genetic algorithms are the right choice in some circumstances. For AI researchers they have the additional advantage of seeming inherently less contrived (i.e. not 'cheating'), even though in practice they may have effective prior information encoded into the huge host of 'tuning parameters'.

[Kwizard]

The massive parallelism of the brain does in fact give it 'obscene' computational resources, but these cannot be directly applied directly to something as symbolic and abstract as written equations, so exactly how intuition contributes to scientific discovery is a hotly contested debate.

What kinds of progress have we made so far in finding out exactly how "deliberative" or "intuitive" symbolic reasoning is?

[Starglider]

What kinds of progress have we made so far in finding out exactly how "deliberative" or "intuitive" symbolic reasoning is?

It's easy to look at the hundreds of plausible theories of brain function and conclude that little progress has been made. However this would be a mistake; the set of credible theories today is much closer together than the set of credible theories in 1980, or the wild guessing still prevalent in 1960. Detailed brain structure analysis and results from fMRI, implanted electrode arrays, cultured neuron studies and physicalist computer simulations are slowly narrowing the field. However the details of how a brain primarily evolved to make intuitive, subconscious and concrete decisions supports abstract thought are likely to be the last details we develop a robust explanation of.

[cosmicalstorm]

Here's another story like the one OP posted, very interesting stuff.

"A science-savvy robot called Adam has successfully developed and tested its first scientific hypothesis, discovering that certain genes in baker's yeast code for specific enzymes which encourage biochemical reactions in yeast, then ran an experiment with its lab hardware to test its predictions, and analyzed the results, all without human intervention. Adam was equipped with a database on genes that are known to be present in bacteria, mice and people, so it knew roughly where it should search in the genetic material for the lysine gene in baker's yeast, Saccharomyces cerevisiae. Ross King, a computer scientist and biologist at Aberystwyth University, first created a computer that could generate hypotheses and perform experiments five years ago. 'This is one of the first systems to get [artificial intelligence] to try and control laboratory automation,' King says. '[Current robots] tend to do one thing or a sequence of things. The complexity of Adam is that it has cycles.' Adam has cost roughly $1 million to develop and the software that drives Adam's thought process sits on three computers, allowing Adam to investigate a thousand experiments a day and still keep track of all the results better than humans can. King's group has also created another robot scientist called Eve dedicated to screening chemical compounds for new pharmaceutical drugs that could combat diseases such as malaria.

http://hardware.slashdot.org/article.pl ... 03/0049237