Repost: Type I and Type II errors: p-values

May 21 2018 Published by under Uncategorized

But really only about type I. I know most everyone knows this. But I remember when I didn't and had to learn it. Maybe this will help.


Type 1 and type 2 are lousy names. They convey no information about what they are. Once, I asked a very good friend who was finding new species and naming them to name if he would be willing to name one after me (I was younger and brasher). He said “hell, no, that name will convey nothing about the species” as in H. longifolia or H. bigantleria

Anyway type 1 error is the type you are always seeing associated with p=.05. It is the probability of being wrong if you reject the null hypothesis. That is, if you say that red fish are bigger than blue fish, based on your samples of each, you could see this much difference in size between them just by chance. You roll a single die several times and get the following distributions – these are the number of times you rolled each number.

Die 1:

density x plot

This is about what you would expect for 100 roles – about equal numbers of each. Now, here is what your labmate (the one who is stealing your data, you think) got, with hir own die:

density y plot

This is NOT what you expect. All those 6’s? nearly 50% 6’s? Now, of course it is possible that someone would roll a die 100 times and get a 6 47 times. But how likely??? That is the question that p-values answer.

In the olden days, when I was young, you had to calculate this crap by hand, or look it up in a table that never ever made sense. But now, with the advent of modern society, decafe coffee, no smoking in restaurants, it is possible to have it computed for you. That is the p-value. What is the probability of seeing this by chance?  p=0.05 tells you how much chance. Or 1 in 20 times you would see this much difference by chance.

And yes, someone is stealing your data (p = 0.00000012 in the above case).

No responses yet

Leave a Reply