An unexpected probability fact

I think your intuition that the boundary should be at 0.625 is a good one, and the fact that the actual boundary is very close to that confirms the intuition.  But it would be surprising if the boundary was exactly 0.625 because we are not dealing with a linear relationship here.

Let's suppose m+n = 176.  I chose this because it means when m = 110, we get exactly m / (m+n) = 0.625.

The fair coin would on average flip 88 heads (ie 0.5 * 176), and the biased coin would on average flip 132 heads (ie 0.75 * 176).  110 is exactly halfway between the two (your "intuition point").

Now imagine any sequence that involves flipping 110 heads and 66 tails.  On the fair coin this has probability 0.5^176 and on the biased coin this has probability 0.75^110 * 0.25^66.  (There are 176-choose-110 such sequences but they all have the same probability so we can focus on one).  These two probabilities are not the same (which is effectively what your intuition was claiming) - the latter is much smaller.

This does make sense, because the biased coin has a smaller standard deviation in its distribution.  (To get a sense of this, imagine as a coin gets more and more biased to heads, it will more and more tend to get close to the *expected* number of heads - ultimately, a coin which shows heads with 100% probability will have zero standard deviation in the number of heads it generates).  This means that the biased coin has a smaller probability of deviating 22 heads from its mean (110 is 22 below the biased mean of 132) than the fair coin has of deviating 22 heads from its mean (110 is 22 above the fair mean of 88).

It turns out that if you consider 111 heads, then you get much closer:

0.5^176 = 1.044 * 10^-53

0.75^111 * 0.25^65 = 0.995 * 10^-53

111/176 = 0.6307, close to your analytical result.

------------------------

Let's re-run my example above but double the number of flips to 352.

It turns out that if we double 111 to 222 and calculate the probability there, we again get close to the two distributions matching, ie

0.5^352 is very similar to 0.75^222 * 0.25^130 .

This is not too surprising because 0.5^352 is simply the square of 0.5^176 and 0.75^222 * 0.25^130 is the square of 0.75^111 * 0.25^65 .   So if we are picking results close to the sweet spot of 0.6309, then the ratio is close to 1 and if we double the number of flips the ratio is squared and we remain close to 1.  So the boundary case of m / (m+n) should be the same however large m+n gets.
It’s quite obvious the concentration of the two distributions are different, why would expect them to meet in the center?
It seems more obvious if you push the extreme all the way. Suppose you have a fair coin and a coin that comes up heads 100% of the time. The midpoint is 75%, but that can't be right because even if you get any tails at all, you cannot be dealing with the biased coin.
If I understand the problem correctly, I think it just comes down to the fact that the binomial distribution is not translationally invariant. The two distributions' pdfs (for P(data | coin #1) vs P(data | coin #2) ) are not just shifted copies of eachother - they actually have different shapes, which means that their ratio as a function of "data" is not going to be 1 halfway between their means.
by

0 like 0 dislike