Tech Notes

My notes on Statistics, Big Data, Cloud Computing, Cyber Security

Tag Archives: type I error

Prediction Study Design

Steps in building a prediction
1. Find the right data
2. Define your error rate
3. Split data into:

  • Training
  • Testing
  • Validation (Optional)

4. On the training set pick features
5. On the training set pick prediction function
6. On the training set cross-validate
7. If no validation – apply 1x to test set
8. If validation – apply to test set and refine
9. If validation – apply 1x to validation Read more of this post

Hypothesis Testing

It is based on the idea that we can tell things about the population based on a sample taken from it.

5 Steps

  1. Hypothesis
  2. Significance
  3. Sample
  4. P-Value
  5. Decide

Inferential Statistics is based on the premise that you cannot prove something to be true, but you can disprove something by finding an exception.

You decide what you want to find evidence for (H1 – there is an effect), ie the alternative hypothesis, then set up the null hypothesis (H0 – there is no effect) and find evidence to disprove it.

This is a statistical method for testing whether the factor we are talking about has any effect on our observation

In other words, this helps us decide if

  • We should believe that the relationship we found in our sample is the same as the relationship we would find if we tested the population
  • OR We should believe that the relationship we found in our sample is a coincidence due to sampling error

Read more of this post