View source for Statistics for Decision Makers - 14.01 - Regression

{{Cat|Statistics for Decision Makers|14.01}}


<slideshow style="nobleprog" headingmark="。" incmark="…" scaled="false" font="Trebuchet MS" footer="[http://www.NobleProg.co.uk www.NobleProg.co.uk]" subfooter="Training Courses Worldwide">
;title: 14.01 - Regression
;author: Bernard Szlachta (NobleProg Ltd) bs@nobleprog.co.uk
</slideshow>



==Simple Regression。==
;Simple linear regression
: Predicts scores on one variable from the scores on a second variable
;Criterion variable
: The variable we are predicting, referred to as Y
;Predictor variable
: The variable we are basing our predictions on, referred to as X

* When there is only one predictor variable, the prediction method is called '''simple regression'''
* In simple linear regression, the predictions of Y when plotted as a function of X form a straight line

==Simple Regression Example。==
{| class="wikitable" style="text-align: center; background-color: white;"
|-
| rowspan="6" | [[File:ClipCapIt-140603-213413.PNG|350px]]
! style="width:50px;"| X
! style="width:50px;"| Y
|-
| 1.00
| 1.00
|-
| 2.00
| 2.00
|-
| 3.00
| 1.30
|-
| 4.00
| 3.75
|-
| 5.00
| 2.25
|}
* There is a positive relationship between X and Y
* We want to predict Y from X
* The higher the value of X, the higher your prediction of Y

==Linear regression。==
[[File:ClipCapIt-140603-213934.PNG]]

*Linear regression consists of finding the best-fitting straight line through the points
*The best-fitting line is called a '''regression line'''

==The error of prediction。==
The error of prediction for a point is the value of the point minus the predicted value (the value on the line)


;Example
{| class="wikitable" style="text-align:right; background-color: white;"
|-
! style="width:80px;"|X
! style="width:80px;"|Y
! style="width:80px;"|Y'
! style="width:80px;"|Y-Y'
! style="width:80px;"|(Y-Y')<sup>2</sup>
|-
| 1.00
| 1.00
| 1.210
| -0.210
| 0.044
|-
| 2.00
| 2.00
| 1.635
| 0.365
| 0.133
|-
| 3.00
| 1.30
| 2.060
| -0.760
| 0.578
|-
| 4.00
| 3.75
| 2.485
| 1.265
| 1.600
|-
| 5.00
| 2.25
| 2.910
| -0.660
| 0.436
|}
*The predicted values (Y') and the errors of prediction (Y-Y')
*The first point has a Y of 1.00 and a predicted Y of 1.21. Therefore its error of prediction is -0.21.

=Regression Line=
==The Best Fitting Line。==
:[[File:ClipCapIt-140604-101920.PNG]]

* The best fitting line is usually the line that '''minimizes the sum of the squared errors''' of prediction 
* The last column in the previous table shows the squared errors of prediction 
* The sum of the squared errors of prediction shown in the previous table is lower than it would be for any other regression line
* This method is called Ordinary Least Squares [OLS]

==The Formula for a Regression Line。==
The formula for a regression line
 Y' = bX + A
 where Y' : predicted score, b  : slope of the line, A  : Y intercept

;Example
The equation for the line in the previous graph is
 Y' = 0.425X + 0.785

* For X = 1, Y' = (0.425)(1) + 0.785 = 1.21
* For X = 2, Y' = (0.425)(2) + 0.785 = 1.64
[[File:ClipCapIt-140603-213934.PNG|ClipCapIt-140603-213934.PNG]]

==The Slope of the Regression Line。 ==
The slope (b) can be calculated as follows:
 b = r s<sub>Y</sub>/s<sub>X</sub>


The intercept (A) can be calculated as
 A = M<sub>Y</sub> - bM<sub>X</sub>


For these data, 
 b = (0.627)(1.072)/1.581 = 0.425
 A = 2.06 - (0.425)(3)=0.785

==Example。==
How could we predict a student's university GPA if we knew his or her high school GPA?
*The correlation is 0.78 
The regression equation is 
 GPA' = (0.675)(High School GPA) + 1.097
A student with a high school GPA of 3 would be predicted to have a university GPA of
 GPA' = (0.675)(3) + 1.097 = 3.12

The graph shows that here is a strong positive relationship between University GPA and High School GPA
:[[File:ClipCapIt-140603-221400.PNG|400px]]

==Assumptions。==
* It may surprise you, but the calculations shown in this section are assumption free
* If the relationship between X and Y is not linear, a different shaped function could fit the data better 
* '''Inferential statistics in regression''' are based on '''several assumptions'''

=Quiz。=
[[Statistics_for_Decision_Makers_-_14.01_-_Regression#Quiz|Please find the quiz here]]

=Quiz=
<quiz display=simple >
{ The formula for a regression equation is

[[File:ClipCapIt-140603-222649.PNG]]

What would be the predicted Y score for a person scoring 4 on X? 

|type="{}"}
{ 10 }

{
{{Show Answer|
10

Plug X equals 4 into the equation to find that Y' equals 3(4) - 2, which equals 10. 
}}
}


{ Suppose it is possible to predict a person's score on Test B from the person's score on Test A. The regression equation is:

[[File:ClipCapIt-140603-222717.PNG]]

What is a person's predicted score on Test B assuming this person got a 40 on Test A? 

|type="{}"}
{ 101.5 }

{
{{Show Answer|
101.5

Plug A equals to 40 into the equation to find that B' equals 2.3(40) + 9.5, which equals 101.5 }}
}


{ Suppose a person got a score of 32.5 on Test A and a score of 95.25 on Test B. Using the same regression equation as in the previous problem, 

[[File:ClipCapIt-140603-222717.PNG]]

what is the error of prediction for this person? 

|type="{}"}
{ 11 }

{
{{Show Answer|
11

The predicted value of B' equals 2.3(32.5) + 9.5, which is 84.25; Error of prediction is B - B', which equals 95.25 - 84.25, which equals 11. 
}}
}


{What is the most common criterion used to determine the best-fitting line? 

|type="()"}
-The line that goes through the most points
-The line that has the same number of points above it as below it
+The line that minimizes the sum of squared errors of prediction

{
{{Show Answer|
The line that minimizes the sum of squared errors of prediction

The most common criterion used to determine the best-fitting line is the line that minimizes the sum of squared errors of prediction. This line does not need to go through any of the actual data points, and it can have a different number of points above it and below it. 
}}
}

{ The mean of X is 3 and the mean of Y is 7. Does the regression line that predicts Y from X must go through the point (3,7)? 

|type="()"}
+True
-False

{
{{Show Answer|
The line that minimizes the sum of squared errors of prediction

Someone who scored the mean on X would be predicted to score the mean on Y. 
}}
}


{  You want to be able to predict a woman's shoe size from her height. You have gathered this information from your female classmates. The mean height of women in your class is 64 inches, and the standard deviation is 2 inches. The mean shoe size is 8, and the standard deviation is 1. The correlation between these two variables is .5. What is the slope of the regression line? 

|type="()"}
-0.00
+0.25
-0.50
-0.10

{
{{Show Answer|
0.25

b is r(sY/sX). 0.5 * (1/2) equal to .25 }}
}

</quiz>