IB Class Notes

Interpreting Correlation

Aims: This note addresses the following:

  1. What does the coefficient of correlation (r) mean?
  2. Is the value of r statistically meaningful?
  3. Worked Examples.
  4. How should I report my results?

What does the coefficient of correlation (r) mean?

  1. Correlation does not means causality. That is, if we find that size of feet and IQ are positively related then this does not mean that size of feet causes higher IQ. \[ r = \large \frac{\sum{xy} - \frac{\sum{x}\sum{y}}{n} }{ \sqrt{ \left[ \sum{x^2} - \frac{(\sum{x})^2}{n} \right] \left[\sum{y^2} - \frac{(\sum{y} )^2}{n} \right] } } \]

  2. The numerator (the upper part of the ratio) measures how much x and y vary together (using the deviations of each from their respective means). The denominator (the lower part of the ratio) is a measure of x and y vary separately.

  3. Product-moment correlation coefficient is another name for r. Another way to express r is
    \( r = \large \frac{S_{xy}}{S_x S_y} \) where
    \( S_{xy} = \sum{xy} - \large \frac{\sum{x}\sum{y}}{n} \),
    \( S_x = \sqrt{ \left[ \sum{x^2} - \large \frac{(\sum{x})^2}{n} \right] } \) and
    \( S_y = \sqrt{ \left[ \sum{y^2} - \large \frac{(\sum{y})^2}{n} \right] } \).

    Note that
    \( S_{xy} = ns_{xy} \) or \( n \)Covariance of \(x\) and \(y\). Distinguish the use of capital \(S\) from small \(s\).
    The covariance of \(x\) and \(y\), \(s_{xy} = \frac{\sum{xy} - \large \frac{\sum{x}\sum{y}}{n} }{n}\)
    Similarly \( S_x = ns_x \) or \( n \)standard deviation of \(x\).
    Standard deviation of \(x\), \( s_x = \sqrt{ \frac{\left[ \sum{x^2} - \large \frac{(\sum{x})^2}{n} \right]}{n} } \) and
    \( S_y = ns_y \) or \( n \)standard deviation of \(y\).
    Standard deviation of \(y\), \( s_y = \sqrt{ \frac{\left[ \sum{y^2} - \large \frac{(\sum{y})^2}{n} \right]}{n} } \)

  4. We need to beware of spurious correlation in a scatter diagram. Although there may appear a relationship on the diagram and mathematically, this does not imply a relationship in reality. For example in a particular city, we find an increase in vandalism of public utilities (like public toilets and telephones) and also an increase in the number of stores supplying organic vegetables over time. In this case, it will be unreasonable to suggest that more organic vegetable stores and vandalism are related.

  5. A rule of thumb for interpreting the \(r\) value is by using this table.

    \(r\) value Strength of (linear) Correlation
    \( 0.75 < |r| \le 1 \) Strong
    \( 0.5 < |r| \le 0.75 \) Moderate
    \( 0.25 < |r| \le 0.50 \) Weak
    \( 0 < |r| \le 0.25 \) Very Weak
    Source: Buchanan & etl. in Maths SL Course Companion, 2012.

UP

Is the value of my coefficient of correlation (r) statistically meaningful?

  1. In the Excel example with weight and height, we obtained an r = 0.94 (to 2 significant figures). Well, is this value of r = 0.94 statistically meaningful? What is the probability that this value is product of random chance? Or in a set of data we obtained an r = 0.500. Does this means a positive correlation or no correlation? How do we decide?
    To decide we need to consult the following table for r.

    v = n-2 degrees of freedom

    p=0.05

    p=0.025

    p=0.01

    p=0.005

    1

    0.9877

    0.9969

    0.9995

    0.9999

    2

    0.9000

    0.9500

    0.9800

    0.9900

    3

    0.8053

    0.8783

    0.9343

    0.9587

    4

    0.7292

    0.8113

    0.8822

    0.9172

    5

    0.6694

    0.7544

    0.8329

    0.8745

    6

    0.6215

    0.7067

    0.7887

    0.8343

    7

    0.5822

    0.6664

    0.7497

    0.7977

    8

    0.5493

    0.6319

    0.7154

    0.7646

    9

    0.5214

    0.6020

    0.6850

    0.7348

    10

    0.4972

    0.5759

    0.6581

    0.7079

    11

    0.4761

    0.5529

    0.6338

    0.6835

    12

    0.4575

    0.5323

    0.6120

    0.6613

    13

    0.4408

    0.5139

    0.5922

    0.6411

    14

    0.4258

    0.4973

    0.5742

    0.6226

    15

    0.4123

    0.4821

    0.5577

    0.6054

    16

    0.4000

    0.4683

    0.5425

    0.5897

    17

    0.3887

    0.4555

    0.5285

    0.5750

    18

    0.3783

    0.4437

    0.5154

    0.5614

    19

    0.3687

    0.4328

    0.5033

    0.5487

    20

    0.3597

    0.4226

    0.4920

    0.5367

    25

    0.3232

    0.3808

    0.4450

    0.4869

    30

    0.2959

    0.3494

    0.4092

    0.4487

    35

    0.2746

    0.3246

    0.3809

    0.4182

    40

    0.2572

    0.3044

    0.3578

    0.3931

    45

    0.2428

    0.2875

    0.3383

    0.3721

    50

    0.2306

    0.2732

    0.3218

    0.3541

    60

    0.2108

    0.2500

    0.2948

    0.3248

    70

    0.1954

    0.2318

    0.2736

    0.3017

    80

    0.1829

    0.2172

    0.2565

    0.2829

    90

    0.1725

    0.2049

    0.2422

    0.2673

    100

    0.1638

    0.1946

    0.2300

    0.2540

    UP

  2. Actually, we will learn how to do a significance test for correlation coefficient r in this note.
    Assumptions
    To carry out this test, we assume that X and Y variables are jointly normally distributed with correlation coefficient ρ.
    ρ is read as “rho” and we can think of this as the (true) population correlation coefficient. We also assume that the observed data consist of a random sample from all possible values (populations) of X and Y.

  3. Forming Null Hypothesis
    The null hypothesis Ho is always that the correlation coefficient r = 0, that is there is no correlation between our variables.
    This null hypothesis is expressed as
    Ho: ρ = 0.

  4. The number of degrees of freedom is total number of observations (n) minus two. If we have ten observations (n=10) then the degrees of freedom is thus 8.

  5. Forming Alternative Hypothesis (I)
    If we have reason to believe that r is positive say from observing a scatter plot of the data (upward trend) then our alternative hypothesis is
    H1: ρ > 0.


  6. Carrying out the test
    The critical value at k% significance level and v degrees of freedom is c. For a 5% significance level, we usually write " ? = 0.05 " . The value ? is the probability of wrongly rejecting the null hypothesis when it is actually true. If we take 5% significance level and 8 degrees of freedom then from the table above our critical value is approximately 0.5493. The critical region is the region in which all r is larger than 0.5493 as illustrated above. For this particular example, the p-value (or the probability to the right of c) for the shaded region is 0.05.

    If our calculated r ,say r = 0.981, is in the critical region then we will reject the null hypothesis Ho of no correlation and accept the alternative that the correlation is positive. We write, “Since r = 0.981 is more than the critical value of 0.5493 at 5% significance level and 8 degrees of freedom then we will reject the null hypothesis and accept the alternative. We conclude that there is statistical evidence to suggest that there is a positive correlation between these data.”
    If our calculated r is NOT in the critical region, say r = 0.305, then we write, “Since r = 0.305 is less than the critical value of 0.5493 at 5% significance level and 8 degrees of freedom then we failed to reject the null hypothesis and reject the alternative. We conclude that there is not enough evidence to suggest that there is any correlation between these data.”

    I suggest that you use the above wordings in bold face with appropriate adjustments for your conclusion. Make sure you always express your conclusion in words and in the context of your task.

  7. Forming Alternative Hypothesis (II)

    If we have reason to believe that r is negative say from observing a scatter plot of the data (downward trend) then our alternative hypothesis is
    H1: ρ < 0.

  8. Carrying out the test
    The critical value at k% significance level and v degrees of freedom is c. For instance if we take 1% significance level and 35 degrees of freedom then from the table above the value is 0.3809. Since our alternative is negative correlation then our critical value is -0.3809.
    The critical region is the region in which all r is less than 0.3809 as illustrated below. For this particular example, the p-value for the shaded region is 0.01.


    If our calculated r ,say r = -0.686, is in the critical region then we will reject the null hypothesis Ho of no correlation and accept the alternative that the correlation is positive. We write, “Since r = -0.686 is less than the critical value of -0.3809 at 1% significance level and 35 degrees of freedom then we will reject the null hypothesis and accept the alternative. We conclude that there is statistical evidence to suggest that there is a negative correlation between these data.”
    If our calculated r is NOT in the critical region, say r = -0.200, then we write, “Since r = -0.200 is larger than the critical value of -0.3809 at 1% significance level and 35 degrees of freedom then we failed to reject the null hypothesis and reject the alternative. We conclude that there is not enough evidence to suggest that there is any correlation between these data.”

      Both alternatives above are examples of one-taileded test. For most practical investigations, we will like to establish whether or not there is a positive or negative correlation then we will be carrying a one-taileded test as above.

  9. Forming Alternative Hypothesis (III)
    If we are only interested in knowing whether or not there is a correlation regardless of sign then we will carry out a two-taileded test. In this case, the alternative is
    H1: ρ ≠ 0.


  10. Carrying out the test
    For instance if we take 5% significance level and 20 degrees of freedom then the critical value at the upper tail is only p=0.025. We need to look at the column with p=0.025 and 20 degrees of freedom in the table. From the table above our critical value is 0.4226. The critical value at the lower tail is thus -0.4226. Each critical region above has a p-value of p/2 or 0.025 in this example.
    The critical region is the region in which all r is either larger than 0.4226 or less than -0.4226 as illustrated above. For this particular example, the p-value for the shaded region is 0.025 for a combined p-value of 0.05.

    If our calculated r ,say r = 0.500 or r = -0.456, is in the critical region then we will reject the null hypothesis Ho of no correlation and accept the alternative that there is some correlation between the two variables. We write, “Since r = -0.456 is less than the critical value of -0.456 at 5% significance level and 20 degrees of freedom then we will reject the null hypothesis and accept the alternative. We conclude that there is statistical evidence to suggest that there is some correlation between these data.”
    If our calculated r is NOT in the critical region, say r = 0.305, then we write, “Since r = 0.305 is less than the critical value of 0.456 at 5% significance level and 8 degrees of freedom then we failed to reject the null hypothesis and reject the alternative. We conclude that there is not enough evidence to suggest that there is any correlation between these data.”

  11. Let say, for a particular sample we have 47 as our degrees of freedom. This value is not in our table and we will use the tabulated values in row v = 45. That is, we will impose a more stringent test.

  12. Notice that the critical value for 100 degrees of freedom (or n = 102) and 0.5 % significance level is only 0.2540 which is less than 0.30. If you have a scatter diagram with total observations more than 102 with r > 0.30 then according to the table above, we have strong statistical significance for this coefficient of correlation.

UP

Worked Examples

Example1
Let us look at our previous example (Excel file with Macro).

X height

Y weight

178

53

174

50

180

58

182

60

190

70

195

85

165

45

168

48

173

51

175

60

x=1780

y=580


(a) Calculate the correlation coefficient r of the data given above.
A researcher believes that people who are taller tend to be heavier in weight.
(b) Assuming that these data are jointly normally distributed with correlation coefficient ρ and the data are collected from a random sample, carry out a significance test at 5% significance level to verify the researcher's belief.
(c) Repeat the test at 1% significance level.

Solution

(a)\( r = \large \frac{S_{xy}}{S_xS_y} \)

To calculate r manually, we need the values for ∑ x , ∑ y, ∑ xy, n, ∑ x2, and ∑ y2.
Here, we are going to use GDC to help us find the above values.
From GDC as shown in the second column,
∑ x = 1780, ∑ y = 580,
∑ x2 = 317612, ∑ y2 = 34928,
∑ xy =104181, n = 10

Sxx = [317612- (1780)2/10] = 772


S
yy = [34928- (580)2/10] = 1288
S
xy = [104181- (1780)(580)/10] = 941
r = S
xy / [ √(Sxx)√(Syy) ]
r = 941 / [ √(772)√(1288) ]
[Note that we have not used an approximation like S
x27.785 to do our calculation because this will reduce the accuracy of the final answer. It is a better idea to enter all the three expressions above into our GDC for the final answer.

r ≈ 0.944 (3 s.f.)

The value of r is positive which indicates that the researcher's belief may be right. .

With the help of GDC.

  1. Press [STAT] and select 1:Edit.

  2. If there are old data in list L1 and L2 then we need to clear these by the following:

    (i) Press [STAT].
    (ii) Use the arrow to move down to 4:ClrList and press [ENTER]
    (iii) Type in [2nd][1] for List 1, [,] [2nd][2] for List 2 and [ENTER].
    (iv) Repeat step 1 and step 2 above.

  3. Enter the values of x into L1 and the values of y into L2.

  4. Press [2nd][MODE] to quit. Press [STAT] and use the arrow button to move across to CALC as below.
    Select 2:2-Var Stats as below, enter L1, L2 and press [ENTER].

    See More on Linear Regression to obtain r with GDC without calculation.


(b) State Ho and H1.
Ho : ρ = 0 ; there is no correlation between height (X) and weight (Y).
H
1 : ρ > 0 ; there is a positive correlation between height (X) and weight (Y)
(i.e. people who are tall tend to be heavier in weight).

State the type of test and its significance level.
We will perform an upper tail significance test at the 5% level.

State the rejection criterion.
Since the sample size is 10 there are 8 degrees of freedom in these data and at 5% significance level we will be looking at p = 0.05 in the table above. The critical value associated with p = 0.05 at 8 degrees of freedom is 0.5493.
We will reject H
o in favour of H1 if r > 0.5493.

Calculate r.
This calculation has been done in (a) above. r ≈ 0.944 (3 s.f.).

Conclusion of the test.
Since r = 0.944 is more than the critical value of 0.5493 at 5% significance level and 8 degrees of freedom. We will reject the null hypothesis and accept the alternative.
We conclude that there is statistical evidence to suggest that there is a positive correlation between these data.

(c) At 1% significance level, the critical value is 0.7154.
Verify this value using the table above.
Since r = 0.944 is more than the critical value of 0.7154 at 1% significance level and 8 degrees of freedom. We will reject the null hypothesis and accept the alternative.
We conclude that there is statistical evidence to suggest that there is a positive correlation between these data, i.e. people who are tall tend to be heavier in weight.


Example 2
Nine students took part in a singing competition in which their scores from an experienced judge and a trainee judge were recorded.

Experienced judge (X)

5.70

6.02

5.10

7.01

8.52

4.52

8.02

5.08

6.67

Trainee judge (Y)

6.33

6.28

7.82

7.34

7.85

5.52

7.08

4.79

6.86

(a) Calculate the product-moment correlation coefficient of the data above.
(b) Use a 5% significance level to test whether or not there is any correlation between the scores given by these judges.
(c) Repeat the above test for 10% significance level.

Solution

(a)\( r = \large \frac{S_{xy}}{S_xS_y} \)

To calculate r manually, we need the values for ∑ x , ∑ y, ∑ xy, n, ∑ x2, and ∑ y2.
Here, we are going to use GDC to help us find the above values.


∑ x = 56.64, ∑ y = 59.87,
∑ x2 = 371.517, ∑ y2 = 406.7583,
∑ xy =383.9254, n = 9

Sxx = [371.517- (56.64)2/9] = 15.0626


Syy = [406.7583- (59.87)2/9] ≈ 8.48975556
Sxy = [383.9254- (56.64)(59.87)/9] ≈ 7.143533333
r = Sxy / [ √(Sxx)√(Syy) ]
r ≈ 7.143533333 / [ √(15.0626)√(8.48975556) ]
r ≈ 0.632 (3 s.f.)

 

(b) State Ho and H1.
Ho : ρ = 0 ; there is no correlation between the scores of these two judges.
H
1 : ρ ≠ 0 ; there is some correlation between the scores of these two judges.


State the type of test and its significance level.
We will perform a two-tailed significance test at the 5% level.

State the rejection criterion.
Since the sample size is 9 there are 7 degrees of freedom in these data and at 5% significance level with two tails, we will be looking at p = 0.025 in the table above. We use p =0.025 because each tail only has 2.5%. The critical value associated with p = 0.025 at 7 degrees of freedom is 0.6664.
We will reject H
o in favour of H1 if r > 0.6664.

Calculate r.
This calculation has been done in (a) above. r ≈ 0.632 (3 s.f.).

Conclusion of the test.
Since r = 0.632 is less than the critical value of 0.6664 at 5% significance level and 7 degrees of freedom. We failed to reject the null hypothesis.
We conclude that there is not enough statistical evidence to suggest that there is a correlation between the scores given by these judges.

(c) At 10% significance level, the critical value is 0.5822 since each tail now has 5% or p = 0.05.
Verify this value using the table above.
Since r = 0.632 is more than the critical value of 0.5822 at 10% significance level and 8 degrees of freedom. We will reject the null hypothesis and accept the alternative.
We conclude that there is statistical evidence to suggest that there is some correlation between the scores given by these two judges.

Here is an example where we make different conclusions given different significance level. If we are conservative and not willing to take more than 5% chance of rejecting the null hypothesis when it is actually true then we conclude that the scores given by the trainee judge is not correlated with those given by the experienced judge. If we are willing to take up to 10% chance of rejecting the null hypothesis when it is actually true then we conclude that their scores are correlated in some way.

UP

How should I report my results?

  1. Report the value of r.

  2. Report the degrees of freedom (v) and compare it with relevant tabulated value in the above table.
    Example: r = 0.94
    Degree of freedom is 8. The significance level is less than 0.5%.
    This report implicitly suggests a one-tailed
    (upper tail) significance test since the calculated value of r is positive, and there is less than 0.5% chance of rejecting the null hypothesis of no correlation when it is actually true.

Email KokMing Lee
Li Po Chun United World College of Hong Kong,
10 Lok Wo Sha Lane, Sai Sha Road,
Shatin, New Territories, Hong Kong SAR.