Hypothesis Testing Explained!
Hypothesis Testing Explained!

Let us try and understand the widely used concept of hypothesis testing and p-values.

Designed using Canva.

Let us consider you are asked to test whether the newly created navigation bar on the home page, of your company website, is attracting more users than before. How will you approach this problem? How can you be sure this change in traffic to your website is due to the addition of navigation bar and not something that happened by coincidence! This is where hypothesis testing comes into the picture.

A Simple Example

Designed using Canva.

Let us consider 4 friends A, B, C, and D who are in a dilemma of choosing the person who cooks every day. So they write their names on 4 chits and pick one chit every day. The person having his name on the picked chit is the person who is going to cook on that particular day. After 4 days, person A did not get a chance to cook. The friends got suspicious and tried to find the probability of person A not getting a chance to cook for 4 days which would be

3/4 * 3/4 *3/4 *3/4 = 81/256 = 0.31 = 31%(since 3/4 is the probability of person A not being picked on one particular day ie. out of 4 total outcomes there are 3 outcomes in which person A is not picked)

31% chance of not being picked for 4 days consecutively seemed fine for the friends and hence they continued the same way. But after 16 days, Person A still did not get a chance to cook. Now the friends again decided to find the probability of person A not getting a chance to cook for 15 days which would be (3/4 multiplied by itself 15 times) = 0.01 = 1%.

Now the friends really got suspicious about person A not getting a chance to cook because the probability is only 1% (which is less than 5%) and concluded that person A has committed to cheating.

Okay….Why did I mention less than 5% in the brackets?

In general, statisticians set a threshold value for things to occur by chance. If the probability is less than that threshold value they conclude that there is no chance for random occurrence. Like in the case of the 4 friends, after 4 days the case of A not getting a chance to cook might have occurred randomly and the probability of 31% denotes that. But after 15 days the probability became 1% which significantly reduces the case of A not getting a chance to cook being random. Here the 5% value is known as significance value. The 31% value for 4 days and the 1% value for 15 days are known as p-values. Here,

Null Hypothesis — Person A did not get a chance to cook purely due to random chance and did not cheat.

Alternative Hypothesis — Person A did not get a chance to cook not due to random chance and cheated

For the 4 days scenario, since the p-value, 31% is greater than the significance value of 5% we do not reject the null hypothesis.

For the 15 days scenario, since the p-value, 1% is lesser than the significance value of 5% we reject the null hypothesis.
['Saiteja Kura']
['Technology', 'Data Science', 'Hypothesis Testing', 'Statistics', 'Probability']