bayesian seasoning

I was tempted to write something about the Higgs boson since it's been in the news again lately (even the birds are against it!), but I thought I'd deviate instead away from PSM and mention briefly a different subject from this semester. Most people have probably have a fairly decent working knowledge of probability - it's difficult to avoid due to its popularity! - and know how to use it, and what it means. On a sidenote, there is actually a sense of theoretical definition behind even that much, such as the significance of higher numbers meaning more likely, and the normalisation over a [0,1] interval.

What you may not be away of is that probability and statistics and all those familiar things have a rival, another set of notions and ideas that challenges the dominance that traditional probability has established in society. This rival is known as Bayesian reasoning. We covered it in a half-course over a month or so, which was just enough time to grasp an introduction to what it is and what this means for data analysis.

One of the main points of Bayesian is the distinction between what we know and what we think. There is on the one hand, what you know about a particular problem you are trying to solve. You might know that a coin can land heads or tails. You might know that rolling two dice has thirty six different outcomes. Then there is what you think - you may think the coin is equally likely to land heads or tails, or you might think that the coin is weighted, so that heads is more likely.

What you think about a certain situation is known as the hypothesis, in Bayesian. It is your hypothesis - the coin is biased, the dice is weighted - that forms your conception of the prior, which consists of the assumptions you are making about your data before your analysis of it. It's a bit tricky, because obviously your prior assumptions can shape what conclusions you draw about the situation. But it is also precisely this characteristic of Bayesian which separates it from traditional forms of probability, because the result is that you no longer say, "The coin will land on heads exactly 50% of the time" (because this is only in the limit of many, many trials). Instead, you give a range, which represents your belief in the likelihood of a particular outcome - "the coin is likely to be heads 60% of the time, with a standard deviation of 5%". Bayesian makes you stop and think about just what kind of beliefs and prejudices you are bringing to your particular problem, and punishes you for either being too neutral or too particular. It's a delicate balance that is difficult to master, but can be very powerful.

The basic ingredients for a Bayesian analysis are then:

1) Hypothesis - the idea or suspicion about something that you want to test. What are the chances this suspect is guilty? Will it rain tomorrow?
2) Prior - what you know about the situation before any further information, the constraints you are able to place on the outcome
3) Likelihood - this incorporates the information you gain from data, the 'new information' you add to the problem
4) Evidence - a normalisation factor which is often neglected depending on the situation
5) Posterior - your final prediction based on all the information available to you, the likelihood of your hypothesis

And to avoid making this note too long, I think I'll leave it about here. I suggest if you're interested to check out the many available sources on the Internet regarding Bayesian - start with Wikipedia and work from there, and you'll gain an understanding of just why Bayesian is becoming increasingly popular as the technology Bayesian calculations often require becomes more and more available. If you're looking for a more comprehensive and pretty good introduction to Bayesian in a more formal way, try any of the books below, they helped me get through the course:

Data Analysis: A Bayesian Tutorial (D. Sivia)
Bayesian Logical Data Analysis for the Physical Sciences (P. Gregory)
Probability Theory: The Logic of Science (E. Jaynes)

One of the really neat things that I (finally) understand enough to write about is Monto Carlo Markov Chain (MCMC) modelling, so I'll write something about that in the near future. It's pretty cool! So keep an eye out for that if you're interested.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License