Tag Archives: statistics

Now you see it, now you don’t

No Planet of Alpha Centauri B?

Last October astronomers announced big news: the discovery of a rocky, scorching hot, Earth-sized planet circling our closest stellar neighbor, the orange dwarf star Alpha Centauri B just 4.3 light-years away. Exoplanet astronomer Debra Fischer (Yale) told the New York Times that the planet next door was the “story of the decade.” Almost lost in the excitement was the caveat that the planet’s detection was still iffy and required heroic efforts to extract any sign of it from the background noise of the star’s radial-velocity measurements.

Now the plot has become more muddled. A new analysis of the data by an independent researcher has failed to confirm the planet’s existence.

How to resolve this issue: Get a lot more data. Everybody involved agrees on that. This is science at work. Finding a planet in the Alpha Centauri system would be really cool (ask any science fiction fan), but we need to be sure it is really there.

Thomas Bayes and O.J. Simpson

After posting about The Prosecutor’s Fallacy I recalled a similar case with the Defense in the O.J. Simpson trial. The issue was summarized in What is your favorite problem for an introduction to probability?:

… one of Simpson’s lawyers, Alan Dershowitz, noted that even though Simpson beat
his wife, that hardly mattered, because in the United States, four million women are
battered every year by their male partners, yet only one in 2,500 is ultimately
murdered by her partner (1 in 1000), so, by the ‘reasonable doubt’ criterion, this is
irrelevant. The jury found that argument persuasive, but it’s spurious. The relevant
question was what percentage of all battered women who are murdered are killed by
their abusers, which ain’t 1 in 1000, but rather 9 in 10.

For a clear explanation of the details see Chances Are, by Steven Strogatz, which is reprinted in his excellent book, The Joy of x: A Guided Tour of Math, from One to Infinity.

The Prosecutor’s Fallacy

Bayes’ Theorem … A Simple Example

Notation: Prob(A) means “the probability of event A” and Prob(A|B) is “the probability of event A, given that event B has happened.”

Bayes’ Theorem: Prob(A|B)xProb(B) = Prob(B|A)xProb(A)

Now, Prob(A|B) and Prob(B|A) are often confused by even the most intelligent of people. The confusion often appears in legal cases and is sometimes called the Prosecutor’s Fallacy. Bayes’ Theorem relates these two distinct conditional probabilities.

Followed by a straightforward example of why this really matters.

Bayesian Probability

Back in 1976, when I got an M.S. in Statistics from Stanford, the dominant interpretation of probability and statistics was the Frequentist view. The alternative Bayesian interpretation was definitely a minority position.

In recent decades the Bayesian view has been gaining ground, especially after the spectacular success of one of its practioners, Nate Silver, in predicting the results of the 2012 U.S. Presidential election. Silver has written an excellent book, The Signal and the Noise: Why So Many Predictions Fail-but Some Don’t, about forecasting. He gives some vivid examples of Bayesian methods.

The main point of Silver’s book is quite clear in the title: Real world data is full of noise. All too often people see some random fluctuation in the data and think that it represents some real pattern. Silver gives examples from many fields, including sports, the stock market, earthquakes, politics, and economics, that show this. In other cases, e.g. weather forecasting and climate change, there is a discernable signal in all of the noise. Silver neatly debunks some of the bad statistical methods used by the deniers of global warning.

Another good book about Bayesian probability is From Cosmos to Chaos: The Science of Unpredictability, by by Peter Coles. Coles assumes a little more comfort with mathematical notation than Silver, but the actual arguments do not require more than algebra. While discussing the history of probability theory from its roots in gambling, he concentrates on physics and astronomy, which also contributed significantly to the development of statistics. He is a strong advocate of Bayesian probability and suggests the Bayesian view avoids some nasty issues in the interpretation of statistical mechanics and quantum mechanics, notably that in
the latter subject there is no reason for the Many Worlds Interpretation. Incidentally, he has also argued that the conventional interpretation of Sherlock Holmes is wrong. See The Return of the Inductive Detective.

The Frequentists vs. Bayesian debate has also made Xkcd. The implication is that some level we are all Bayesians, even if we don’t admit it.

On an issue in reasoning with probabilities, Ethan Siegel discusses the Inverse gambler’s fallacy in The Last Refuge of a Science-Denying Scoundrel.

Statistics Hell

Welcome Mortals To My Domain

You hate statistics, you despise maths, you stain your pants at the mention of sphericity. Normal people love these things, but you, barrel of number-fearing sputum that you are, are terrified. Afraid for your worthless life you leapt pathetically to the internet for guidance. A freak spark of lightening hit your house sending a bolt of electricity through your computer. Sparks flew like tendrils from the screen, fusing with your face and sucking your head and body into the number vortex that is statistics hell. Your crime is evacuating your bowels at the mention of a t-test, your punishment is eternity in statistics hell. I am the gatekeeper, the evil ruler of this world of numbers. Although you might confuse me for a human full of empathy and compassion for those taking their first wobbly steps through this horrific world of equations, underneath my skin I am numbers without a soul.

Another winner on Tuesday

I have a degree in Statistics (M.S., Stanford, 1976). I really enjoyed reading all of this.

Why Math is Like the Honey Badger: Nate Silver Ascendant

If ever there was an iron-clad case to be made for math literacy, it’s what happened over the last few weeks with the New York Times‘ star statistician Nate Silver and his 538 blog (named after the 538 votes in the electoral college).

Nate Silver does it again! Will pundits finally accept defeat?

For the practical implications: How Conservative Media Lost to the MSM and Failed the Rank and File

Also this Tweet:

Karl Rove forms new SuperPAC to run negative ads against “the scourge of math and statistics that’s ruining America”.

Red-Shirt Risk

How Likely Is It That You’ll Die?

Reporting on Matt Bailey’s Analytics According to Captain Kirk, which begins with a quantitative summary of what all Star Trek fans know: “red-shirted crewmembers died more than any other crewmembers on the original Star Trek series.” — 73% of crew deaths.

However, Bailey does not stop there. The rate of red-shirt death varies considerably, and in some shows is much less than in others. Those shows share another well-known feature of the original series, which suggests a risk mitigation strategy:

Continue reading

Higgs articles

Higgs 101. This is a few years old, but still good.

Final Word from the Tevatron on the Higgs Hunt

Live-Blogging the Higgs Seminar

The official Press Release: CERN experiments observe particle consistent with long-sought Higgs boson

The Biggest Firework of them all: The Higgs!

New baby boson is born, weighing in at about 126 GeV

Higgsdependence Day!

Does 5-sigma = discovery? [Yes]. What all those sigmas mean, and the difference between random and systematic error. Also see The Higgs Boson – Certainly, certainly (?) there! (at least, I am pretty certain it is).

There should be a Nobel Prize out of this, but who should get it? This is a non-trivial problem. On the Higgs row and Nobel reform