Math question: How does probability cope with this?

[Apollonius]

Inspired by that joke about G.W. Bush allegedly being shocked when hearing that 50% of all Americans have below average IQs. At the first glance it's funny, because it appears to be a trivial fact that the population is more or less evenly distributed on both sides of the 100 IQ marker. But when you think about it some more, you realise that due to the bell curve shape of the distribution, very many people will be neither below nor above the average, they'll be just average. So far, so good.

But now the problems begin for me. Let's abstract this a little.
- Let's say, we have a line of a defined length. I think the actual length may be irrelevant, but let's just say it's 1 meter long.
- On that line we have a known number of points. They're real points with zero length. They're randomly distributed on the line, and we can't know their position, unless...

OK, forget the above. This would get more complicated than my math illiterate brain could handle. Here's a simpler experiment:
I will think of a random rumber between 1 and 200. It doesn't have to be an integer, it could have as many decimal points as I want. You have to guess that number. You only have one guess.
Now, if I were only allowed to think of integers, your chances of guessing my number would be 0.5%, right? But since I have an infinite er, number of numbers to choose from, your chances of guessing correctly are...
Well, what are they? Zero? One divided by infinity? Is there a practical difference? Is there a theoretical difference? Are we even allowed to divide by infinity, or is it similar to that infamous division by zero?
Now, let's make it even more complicated. Now I'm not thinking of just one number, but of 300,000,000 numbers between 1-200. I'm also telling you that about one third of my thought-up numbers will be between 99-101. Did your chances of guessing a correct number just increase, or are they still the same? How much is [(300 million) x (very little)]?

And as an aside, does this ultimately mean that the above joke is funny after all?

[Kuroneko]

If the distribution in symmetric, then the mean is also the median, in which case 50% of the population will be below average. There is no general reason why the mean and median have to be the same, other than the central limit theorem making it in some sense "reasonable" for many phenomena to be roughly Gaussian (bell curve-shaped). If you didn't know the details of the distribution, particularly whether the mean and the median are equal, the joke isn't funny.

If you're picking random numbers in [1,200] with some continuous probability distribution, the chances that I'll guess the number you picked are exactly zero. If your picking distribution is not continuous--which is, as a practical matter, virtually guaranteed, since we're incapable of picking just any real numbers, but rather just the ones that are computable in some way starting from a few simple numbers (say, by specifying an algorithm to compute it)--then the chances of me getting exactly your number might not be zero. However, since a variety of psychological factors make this related problem not well-defined, let's ignore it.

Again, if your distribution is continuous and you pick a subset of them (300 million or whatever), then the chances of me picking a number in your subset is what's called the probability measure of the subset. Mathematically, the integral of the probability density function over that subset. However, this will always turn out to be zero for any finite or countable infinite subset (and even some, though not all, uncountably infinite ones). In that case, it won't matter if you tell me the proportion of numbers within some sub-interval.

[Jaepheth]

You could also think about it this way:

For me to pick the number you're thinking of, I'd first have to guess the first decimal digit correctly, and I have a 1 in 10 chance of doing that. Then I have to pick the next decimal digit correctly, again 1/10. I have to choose every decimal digit correctly in order to guess the number you're thinking of, and if you're using real numbers there's an infinite number of decimal digits I have to guess correctly. The probability of me doing this is lim x->infinity .1^x (which is 0) for any countable set of numbers.

So we could play the game where:
1. You think of an integer from 0 to 9 and I take a guess at what it is
2. If I guess right: repeat step 1, if not: I lose

Clearly, there is 0 chance of me winning this game because it only ends when I lose.

[Surlethe]

When you're dealing with a continuous distribution, the really only meaningful measure of probability is asking: what are the chances that the number you picked lie between two specified numbers a and b? That gets around the fact that, as you noted, the chances of picking one element out of an infinite (in the case of the real numbers, uncountably infinite) set.

Here's an explanation with basic calculus. What you're describing is a probability density ρ(x) defined on [1,200]. If you pick a point randomly, according to the density, the probability that your point lies between two points a, b is Int_{a}^{b} ρ(x) dx. Obviously, your point is between 1 and 200, so Int_1^200 ρ(x)dx = 1. What does this mean for me picking a random point? Well, that corresponds (more or less) to integrating just over that point -- i.e., if I choose (say) c, then the probability that you chose c is Int_c^c ρ(x)dx = 0.

Unless ρ(x) = δ(x) and c = 0.

[Apollonius]

What? No controversy at all? Bah, disappointing.
When I thought of this problem, it seemed so exciting and mind-boggling to me, yet you made it appear so simple and definite. Damn you!

But I'm not giving up. There's got to be something interesting in this whole thing. What would happen if we extrapolated the fact that the chance of guessing one of the numbers correctly is 0? Let's forget the guessing part. Now I'm only thinking numbers to myself. The chance that I'm going to think of a specific number x is 0, right? So is the probability of my thinking of a number y. Or any other number that we would define.
So, if the chances of every number are equally 0, how come I'm still able to pick one regardless? How can something occur that has 0 probability of occuring?

[fuzzymillipede]

When asking someone to come up with a random number, they might say 3, 26, 2141, 67458458, etc, but they probably won't come up with a 10,000,000x10^10,000-digit number, which according to you, has the same probability of a 2-digit number. Even a computer couldn't come up with every possible number, because of memory constraints. Simply put, it is impossible to comprehend an infinite amount of distinct numbers.

Thus, the probability of a person coming up with a number is not 0, because the person must draw from a finite range of numbers within his capacity to comprehend.

[Apollonius]

Hey, that's cheating! Using common sense in a convincing way is unfair! How am I to argue against that now? Spoilsport.

[Apollonius]

Is there no process in the universe whose number of potential outcomes is infinite? What if I throw a single hydrogen atom at a wall? Will the possible impact points of the atom not be infinite?

[Terralthra]

The number of places it can impact are limited by the area of the wall and the planck length.

[Apollonius]

Is this really the case? I've heard of the Planck length before, but I've always been to intimidated by it to do any further digging. But after reading your reply, I googled it up and read the three first hits (the first being Wikipedia). All three explanations were surprisingly short, while unsurprisingly filled with arcane mathematical symbols that looked at me like they wanted to eat me.

In short, I still don't really know what Planch length actually is. I gather that it is the smallest possible distance that two things can have relative to one another. But if you say that the possible coordinates on my wall are finite, wouldn't that mean that the wall is somehow like a computer screen, made of pixels? Is space really made of pixels? How are they arranged? I don't suppose they're cubes with the diametre of 1 Planck length, since then they would have hypotenuses and er... (whatever the distance that connects two opposite corners of a cube is called) that are not integer multiples of 1PL. My intuition would tell me they're spheres (part of me is almost tempted to ask: "spheres of WHAT?", but the rest of me knows how little sense that would make).

If we mapped all the possible positions of the ideal centre of mass of a hydrogen atom on a two-dimensional area 10 planck lengths high and 10 PL wide, what would that map look like? Would there be neat rows like the pixels on my screen, or would there be a staggered pattern, maybe forming a hexagon pattern? I half expect you to tell me that the question is as meaningless as "what came before the Big Bang?" (Why are all the interesting questions meaningless? The universe is really cruel!)

Does the existence of Planck length also imply that all the existing distances in the universe have to be integer multiples of it? Can something be 500,000,000,000,000,000,000.5 Planck lengths away from something else? (I know that number is probably too small for it to apply to any physical objects, but I can't even grasp the numerical dimension of the PL.)


P.S.: Honestly, I'm not trying to prove a point or push some agenda or construct some joke here. I'm just genuinely curious and stupid.

[Jaepheth]

If I remember correctly it's more like the resolution of physics.

You can't measure any position or time any more precisely than 1 Planck length. So while space-time may be continuous, you can only perceive it in a quantized form.

[Apollonius]

So the possible coordinates within finite space are really infinite?

[Dark Lord of the Bith]

On a similar note, what about picking numbers from the Cantor set? The probability of picking any single number is still 0 because the set is uncountable, but what about uncountable subsets of the Cantor set, since it has a Lebesgue measure of 0?

[Surlethe]

Do you mean the probability of picking an element of the Cantor set out of [0,1], or the probability of picking an element of some uncountable subset U of the Cantor set? I don't have the tools to deal with measure theory (yet ...), but I'd wager that in both cases the probability is zero.

[Dark Lord of the Bith]

Do you mean the probability of picking an element of the Cantor set out of [0,1], or the probability of picking an element of some uncountable subset U of the Cantor set? I don't have the tools to deal with measure theory (yet ...), but I'd wager that in both cases the probability is zero.

I mean a random variable x with a uniform probability distribution P(x) where x is in the Cantor set (C). So given a random number from C, what's the probability that x is in some uncountable subset U of C?

As an example, let U be [0, .5] intersected with C (in other words, the first half of the Cantor set). Then it would naively seem that the chance of x being in U is 1/2, but I'm not sure how to show it. And what if U isn't constructed as nicely? What if U is the set of irrational numbers in C? I have very little clue how to approach that.

[Wyrm]

I think both cases yield to elementary analysis, Surlethe, which at least shows that a uniform distribution does not exist for an uncountable set.

Suppose B is a countably infinite set, and we propose to put a uniform distribution on it (discrete in this case). For a distribution to be uniform, all sets of the same volume have the same probability assigned to them. I don't think I should have to explain that, for a discrete uniform distribution, all individual events (choosing p is the correct number) are have the same probability, c. Is it possible to chose c∈(0,1] such that P(C) = 1, as it must when all possible events are in B?

Clearly not, because if we index the elements of B by i∈{1,2,3...}, P(C) = ∑_{i=1}^∞ P(i) = 1, and by hypothesis, P(i) ≡ c, so ∑_{i=1}^∞ P(i) = ∑_{i=1}^∞ c = lim_{n→∞} c ∑_{i=1}^n 1 = lim_{n→∞} cn. The only choice of c that makes the limit exist and equal 1 is c = 1/n, but lim_{n→∞} c = 0. Therefore, all elements from B are impossibilities. This is a contradiction, so no uniform distribution on the whole of B exists. Discrete distributions on countably infinite sets exist only if the distribution is non-uniform; that is, some possibilities have to be more likely than others.

How does this apply to uncountable subsets like the Cantor set or uncountable subsets of same? The above reasoning works because when n is allowed to become larger than any natural number (that is, when the cardinality of elements considered is allowed to become larger than any finite cardinality), no positive constant c can be found to make the limit exist and equal 1. This will only be compounded when an uncountable infinity is considered (as they are larger than countable infinities).

Dark Lord of the Bith, I think you're going to have to pray to Kuroneko for this one. Your problem is you need to be able to construct a measure for the Cantor set, which is really, really hard given that it has a Lesbegue measure of 0. However, showing that finding an element x∈C∩[0,1/2] only requires you to show that P(C∩[0,1/2]) = P(C∩(1/2,1]), as P(C) = P( (C∩[0,1/2])∪(C∩(1/2,1]) ) = P(C∩[0,1/2]) + P(C∩(1/2,1]) = 2P(C∩[0,1/2]) = 1, therefore P(C∩[0,1/2]) = 1/2.

[Apollonius]

∑_{i=1}^∞ P(i) = ∑_{i=1}^∞ c = lim_{n→∞} c ∑_{i=1}^n 1 = lim_{n→∞} cn.

Oh... My... God... What have I done! How do you guys visualise this stuff? Or don't you have to?
Musicians, for example, hear the notes or chord progressions inside their head, when they write them. I know I do. So when what I have written looks different than what I hear, I know there's been a mistake. But what do you do to manage this stuff in your head? That equation (I'm not even sure it is an equation) looks like Linear A to me.

This is way over my head. I hope it's OK to ask some more dumb questions...
What does uniform distribution mean?
What is the difference between countable and uncountable infinite sets? Isn't infinity always uncountable?
(There are, of course, more things I don't understand, but these two questions look like I could actually have a chance of understanding their answers...)

[Surlethe]

Oh... My... God... What have I done! How do you guys visualise this stuff? Or don't you have to?
Musicians, for example, hear the notes or chord progressions inside their head, when they write them. I know I do. So when what I have written looks different than what I hear, I know there's been a mistake. But what do you do to manage this stuff in your head? That equation (I'm not even sure it is an equation) looks like Linear A to me.

Well, part of it is visualization, part of it is 'speaking the language', so to speak, and part of it is simply understanding the rules of symbol manipulation. We do it the way musicians hear notes or chord progressions inside their heads: years of practice.

This is way over my head. I hope it's OK to ask some more dumb questions...
What does uniform distribution mean?

A uniform probability distribution is the probability density that corresponds to a constant probability.

What is the difference between countable and uncountable infinite sets? Isn't infinity always uncountable?

No! (Isn't that cool?) A countable infinite set is one that you can index with natural numbers -- e.g., {1, 2, 3, 4, 5, 6, 7, ...}, or {apple_1, apple_2, apple_3, ...}. But there are infinite sets that are so much bigger you can't even list them -- the irrational numbers, for example. Cantor's proof of this is pretty neat; I, or someone else here who does math, can recreate it on request.

[madd0ct0r]

for more infinite fun, have a look at solids of revolution.

It's easily possible to create an infinitely long object (looks a bit like a medieval trumpet) with infinite surface area but finite and calculable volume.


In other words, you cannot paint it - it has an infinite area and so you'd be there forever and never have enough paint.

BUT you can pour paint into until it's full quite easily. Thus, once you pour the paint back out, you've painted it.

[Jaepheth]

I think the Grand Hotel is a more entertaining mind fuck. And is fairly relevant to the topic of countably infinite sets.

[Apollonius]

A uniform probability distribution is the probability density that corresponds to a constant probability.

I'm not sure I understand this. After 2 minutes of gazing at that sentence, I think it means that every.. uh.. thing that is possible, is equally so (the graph would be a straight line, while non-uniform would be bell-shaped for example). Is that right?

A countable infinite set is one that you can index with natural numbers

...so far I'm still following you. So "countable" does just mean we can start counting them, it doesn't mean we have to walk the entire line. Right?

But there are infinite sets that are so much bigger you can't even list them

Eh? Why not? For lack of what? Because we can't define them or don't know what they are? But if so, how do we know they are infinite or even exist?

Cantor's proof of this is pretty neat; I, or someone else here who does math, can recreate it on request.

If it involves yet more Olmec hieroglyphs, I'd rather not, thankyouverymuch. [/i]

[Apollonius]

for more infinite fun, have a look at solids of revolution.

It's easily possible to create an infinitely long object (looks a bit like a medieval trumpet) with infinite surface area but finite and calculable volume.


In other words, you cannot paint it - it has an infinite area and so you'd be there forever and never have enough paint.

BUT you can pour paint into until it's full quite easily. Thus, once you pour the paint back out, you've painted it.

Tell me more about this. If it's anything like a trumpet, I might even have a chance of understanding it.
Does it require additional dimensions (in which case I'm outta here, lol), or could it be visualised in 3D space?

[Jaepheth]

Tell me more about this. If it's anything like a trumpet, I might even have a chance of understanding it.
Does it require additional dimensions (in which case I'm outta here, lol), or could it be visualised in 3D space?

He's talking about Gabriel's Horn which is a 3d shape (resembling the bell of a horn) with infinite surface area, but finite volume.

[Wyrm]

A uniform probability distribution is the probability density that corresponds to a constant probability.

I'm not sure I understand this. After 2 minutes of gazing at that sentence, I think it means that every.. uh.. thing that is possible, is equally so (the graph would be a straight line, while non-uniform would be bell-shaped for example). Is that right?

Yes, you're getting the idea. Let me put you on a little more solid ground. Suppose you had a lump of clay, weighing, say, 1 kg. Also, say you had a yardstick, and you must spread the 1 kg of clay along the yardstick any way you want, provided that the entire lump of clay must fit on the yardstick, and the clay must have identical height across the narrow dimension of the yardstick. The amount of clay between two arbitrary tickmarks is directly proportional to the probability of a random variable dictated by that distribution to land between those two values.

A uniform distribution would, between any two tickmarks of the yardstick, have clay equal height across the long dimension of the yardstick, like a block. Non-uniform is any other shape.

A countable infinite set is one that you can index with natural numbers

...so far I'm still following you. So "countable" does just mean we can start counting them, it doesn't mean we have to walk the entire line. Right?

Not quite. You can start counting an uncountable set, but you'll never be able to finish, just like in the countable case. Two finite set are the same size if you can attach strings between members of the two sets such that each member of one set has exactly one string that ties it to a unique member of the other set, and vice versa. We can extend this idea to countable infinities by imagining a schema that ties a string from one member to the other in the same way. If we can do it, the two infinite sets are the same size. If one has left-over members, then that set is bigger (just like in the finite case). The schema that ties each member of one set to a unique member of the other set is a one-to-one correspondence, or bijection.

In this language, a countable infinity is one that has at least one bijection with the natural numbers... that is, for each member of the set, there is one and only one natural number it corresponds with, with no left-over natural numbers. By this, the even numbers are a countable infinity, the correspondance being dividing the even number by 2:

{2,4,6,8...} ⇄ {1,2,3,4...}

But there are infinite sets that are so much bigger you can't even list them

Eh? Why not? For lack of what? Because we can't define them or don't know what they are? But if so, how do we know they are infinite or even exist?

No, the proofs of uncountability are immune to such little details. The proof involves assuming that you can list the set in some manner, then arriving at a contradiction. Basically, you use an arbitrary list, undefined in its details, and use that list to construct of a member that is unambiguously NOT on that list. This gives you a contradiction (the listing is supposedly complete, yet we found a member not on the list), and therefore the original premise that we can make such a list is wrong.

[Winston Blake]

So, if the chances of every number are equally 0, how come I'm still able to pick one regardless? How can something occur that has 0 probability of occuring?

As I understand it, a probability of zero isn't the same as being certain to not happen. It's 'almost certain' to not happen. Consider throwing darts at a dart board (in continuous space, to neutralise the Planck length thing). Assume hitting the board has a uniform probability distribution over its whole area (i.e. pick a point at random).

The probability of hitting the left half of the dart board is 0.5. The probability of hitting any quarter of the board is 0.25. The probability is generally the area you're hitting divided by the total area. So when you make a throw, and you hit a point, the probability of hitting that point is zero. Yet you still hit the point. Similarly, the probability of missing all the other points is 1, yet you still missed them.

1 doesn't mean 'certain to happen' and 0 doesn't mean 'certain not to happen'. They're infinitesimally close to being certain, so they're called 'almost certain'.