Tech Notes

My notes on Statistics, Big Data, Cloud Computing, Cyber Security

Degrees of Freedom

What is the minimum number of observations required to draw the regression for a single independent variable
y = b0 + b1x + e

not 1 – because you cant draw the regression

not 2 – because then R Square = 1 anywhere you put the 2nd point.

Screenshot_112713_110900_PM

3! – Drawing a 3rd point gives the line some “freedom”. So df = 1Screenshot_112713_110650_PM

What is the minimum number of observations required to draw the regression for a single independent variable
y = b0 + b1x1 + b2x2 + e (Eg y=weight, x1= height, x2=mothers height)

For this, instead of a line of best fit, it will be a plane of best fit.

So the number of observations cant be 1,2,3. We need minimum 4 points to provide some freedom to the plane to cut through those 4 points in the best fit.

Screenshot_112713_111601_PM

Formula -> df = n-k-1 (n=no of observations, k – no of explanatory variables)

How is df related to R Square ?

If more and more independent variables to a model – no matter how useless they are,  as a result df decreases (i.e. ) R Square will increase.

R Square can be deceiving when we have low df. Hence adjusted R Square is used – which accounts for the low df. (formula not noted here)

Screenshot_112713_113625_PM

As number of variables increase R Sq keeps increasing , but adjusted R Sq decreases.

With a lesser number of observations, the effect is greater.

So overall, the higher the Adj R Sq, the better the model. So choose the number of variables carefully.

Disclaimer : These are my study notes – online – instead of on paper so that others can benefit. In the process I’ve have used some pictures / content from other original authors. All sources / original content publishers are listed below and they deserve credit for their work. No copyright violation intended.

Referencesfor these notes :

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: