# Linear Regression Model - Hypothesis Testing

EC295

Wilfrid Laurier University

Fall 2022

# Introduction

## Introduction

• We previously learned how to estimate regression parameters

• In this section, we learn how to test claims about those parameters

• Hypothesis testing follows the same steps we learned previously

• Make a claim about the regression parameter (e.g. $\beta_{1} = 0$)

• Use estimate $\hat{\beta}_{1}$ and its sampling distribution to evaluate probable truth of claim

• If claim is unlikely to be true, reject it

• We also explore assumptions about the regression error term

• Affects standard error of $\hat{\beta}_{1}$

• This in turn affects the test statistic in hypothesis tests

# Testing Hypotheses About One Regression Coefficient

## Testing Two-Sided Hypotheses about $\beta_{1}$

• The population regression model was

$Y_{i}= \beta_{0} + \beta_{1}X_{i} + u_{i}$

• Imagine that you are interested in $\beta_{1}$

• This is an unknown parameter

• A feature of the population

• But we do not observe the population

• We estimate $\beta_{1}$ with the Ordinary Least Squares (OLS) estimator $\hat{\beta}_{1}$

• Use this to test claims about $\beta_{1}$

## Testing Two-Sided Hypotheses about $\beta_{1}$

• Follows a set of steps

1. Formulate opposing hypotheses about $\beta_{1}$

2. Choose a test statistic

3. Formulate a decision rule

4. Use sample data and apply decision rule

• Step 1: Opposing hypotheses in a two-sided test would be

• $H_{0}: \beta_{1} = \beta_{1,0}$

• $H_{1}: \beta_{1} \neq \beta_{1,0}$

• $\beta_{1,0}$ is the value of the claim

• In regression, claims are about relationship between $X$ and $Y$

• e.g., if claim is that $X$ and $Y$ are unrelated, then $\beta_{1,0} = 0$

## Testing Two-Sided Hypotheses about $\beta_{1}$

• Step 2: choose a test statistic

• t-statistic Measures distance of estimate away from claim $t = \frac{\hat{\beta}_{1} -\beta_{1,0} }{SE(\hat{\beta}_{1})}$

• This is a random variable, because it varies across samples

• t-statistic has a Standard Normal distribution in large samples

• Result of the Central Limit Theorem
• Main complication is computing $SE(\hat{\beta}_{1})$

• Recall that the variance of $\hat{\beta}_{1}$ is

$\sigma^2_{\beta_{1}}=\frac{VAR\left( (X_{i} - \mu_{X})u_{i}\right) }{n(\sigma_{X}^2)^2}$

## Testing Two-Sided Hypotheses about $\beta_{1}$

• Step 2 continued

• $\sigma^2_{\beta_{1}}$ depends on unknown population variances

• $VAR\left( (X_{i} - \mu_{X})u_{i}\right)$

• $\sigma_{X}^2$

• Replace each with estimators

$\hat{VAR}\left( (X_{i} - \mu_{X})u_{i}\right) = \frac{1}{n-2} \sum_{i=1}^{n} (X_{i} - \bar{X})^2\hat{u}_{i}^2$ $\hat{\sigma}_{X}^2 = \frac{1}{n} \sum_{i=1}^{n} (X_{i} - \bar{X})^2$

## Testing Two-Sided Hypotheses about $\beta_{1}$

• Step 2 continued

• With this, we get the the estimator of the variance of $\sigma^2_{\beta_{1}}$ $\hat{\sigma}^2_{\beta_{1}}=\frac{1}{n}\frac{\frac{1}{n-2} \sum_{i=1}^{n} (X_{i} - \bar{X})^2\hat{u}_{i}^2}{\left[ \frac{1}{n} \sum_{i=1}^{n} (X_{i} - \bar{X})^2 \right ]^2}$

• $SE(\hat{\beta}_{1})$ is the square root of $\hat{\sigma}^2_{\beta_{1}}$

$SE(\hat{\beta}_{1}) = \sqrt{\hat{\sigma}^2_{\beta_{1}}}$

• Function is routinely produced in programs like Stata

• Step 3: Form a decision rule

• Typically, set $\alpha = 0.05$, with corresponding critical value $t^c = 1.96$

## Testing Two-Sided Hypotheses about $\beta_{1}$

• Step 4: Compute t-statistic and apply decision rule

• If $\hat{\beta}_{1}$ is too far from $\beta_{1,0}$ given decision rule, reject $H_{0}$
• Alternatively, use p-value approach

• $p$-value is likelihood of getting $\hat{\beta}_{1}$ further away from $\beta_{1,0}$ than observed value

• for a two-tailed test

$p-value = 2 \times Pr[|t| > |t^{act}|] = 2 \times Pr \left[ \left | \frac{\hat{\beta}_{1} -\beta_{1,0} }{SE(\hat{\beta}_{1})} \right | > \left | \frac{\hat{\beta}_{1}^{act} -\beta_{1,0} }{SE(\hat{\beta}_{1})} \right | \right ]$

• Reject for any $\alpha >$ $p$-value

• As we have mentioned, the most popular two-sided test involves $H_{0}:\beta_{1,0} = 0$

• The p-value for this test is automatically reported in Stata regression output

• Standard error also reported for testing other hypotheses

## Testing One-Sided Hypotheses about $\beta_{1}$

• One sided hypotheses involve inequality constraints

• For upper-tailed tests, $H_{0}: \beta_{1} \le \beta_{1,0}, H_{1}: \beta_{1} > \beta_{1,0}$

• For lower-tailed tests, $H_{0}: \beta_{1} \ge \beta_{1,0}, H_{1}: \beta_{1} < \beta_{1,0}$

• t-statistic is computed in the same way

$t = \frac{\hat{\beta}_{1} -\beta_{1,0} }{SE(\hat{\beta}_{1})}$

• Interpretation of t-statistic is different

• For upper-tailed tests, reject only if t-statistic is large positive

• For lower-tailed tests, reject only if t-statistic is large negative

• As such, critical values are computed only in one tail

## Testing One-Sided Hypotheses about $\beta_{1}$

• $p$-value formula slightly different

• For upper-tailed tests, $p$-value = $Pr[t > t^{act}]$

• For lower-tailed tests, $p$-value = $Pr[t < t^{act}]$

• One-sided hypothesis tests are rare

• Use only when there is an obvious reason

• In regression, generally we are interested in testing “significance” of a variable

• Whether it has an effect at all, positive or negative
• Note we can also test claims about $\beta_{0}$

• Follows same procedure, with different standard error

• We will not cover this in lecture

## Errors in Hypothesis Testing

• Two kinds of mistakes in hypothesis testing

Error Table for Hypothesis Testing
$H_{0}$ True $H_{0}$ False
Accept $H_{0}$ Correct Type II Error
Reject $H_{0}$ Type I Error Correct
• Type I Error: Rejecting a true null hypothesis

• Recall: $t$ assumes $H_{0}$ is true

• Reject claims that are unlikely under sampling randomness

• Though unlikely, these values are possible when $H_{0}$ is true

• We could mistakenly reject $H_{0}$ when it is true if we get an odd sample

## Errors in Hypothesis Testing

• Likelihood of Type I error equals to $\alpha$

• $\alpha$ defines improbable values when $H_{0}$ is true

• Though unlikely, those values actually occur $\alpha \%$ of the time when $H_{0}$ is true

• So, probability of Type I error is $\alpha$

• Why not set $\alpha$ really low to avoid Type I Error?

• Makes it more difficult to reject any hypothesis

• Increases likelihood of Type II Error

• Type II Error: Accepting a false null hypothesis

• Depends on true value of parameter

• So we do not actually know likelihood of Type II Error

## Errors in Hypothesis Testing

• But, we do know that

• As $\alpha \downarrow$, Pr[Type II Error] $\uparrow$

• Higher $\alpha$ means accepting more null hypotheses

• Including false ones

• As $\beta_{1}$ gets closer to $\beta_{1,0}$, Pr[Type II Error] $\uparrow$

• Hard to tell $H_{0}$ from truth when they are very close

• Many values will fall in acceptance region, even if $H_{0}$ is false

• Power: Probability of correctly rejecting $H_{0}$

• Equals $1 -$ Pr[Type II Error]

## P-values

• Another way to do hypothesis testing avoids pre-defining a significance level

• Instead find fraction of values more extreme than the estimate

• Find all significance levels consistent with rejection/acceptance

• P-value:The probability of drawing a test statistic at least as extreme as the one computed from the sample

• Small p-value means few values are more extreme

• Sample estimate is in the tails of the distribution

• Unlikely caused by random sampling, so $H_{0}$ may not be true

• Large p-value means many values could be more extreme

• Sample estimate is closer to the middle of the distribution

• Possible from random sampling , so $H_{0}$ may be true

## P-values

• More formally, p-value is defined as below

• Lower-tailed test: p-value = $Pr[t < t^{act}]$

• Upper-tailed test: p-value = $Pr[t > t^{act}]$

• Two-tailed test: p-value = $2*Pr[t > |t^{act}|]$

• P-values define significance levels consistent with acceptance/rejection

• All $\alpha >$ p-value lead to rejecting $H_{0}$

• All $\alpha <$ p-value lead to accepting $H_{0}$

# Confidence Intervals for Regression

## Confidence Interval for $\beta_{1}$

• We discussed previously that point estimates lack information about sampling uncertainty

• Confidence intervals directly incorporate that information into the estimator

• We can construct confidence intervals for $\beta_{1}$

• Remember that confidence intervals are constructed by adding and subtracting a “margin of error” from the point estimate

$\mbox{CI } = \mbox{Point Estimate } \pm \mbox{ Margin of Error}$

• In the case of regression

• The point estimate $= \hat{\beta}_{1}$

• The margin of error $= t^c \times SE(\hat{\beta}_{1})$

## Confidence Interval for $\beta_{1}$

• So a confidence interval for would $\beta_{1}$ be $\hat{\beta}_{1} \pm t^c \times SE(\hat{\beta}_{1})$

• The interval depends on three things

• $\hat{\beta}_{1}$, which we know from our regression

• $SE(\hat{\beta}_{1})$, which we also know

• A critical value $t^c$

• Same critical value from a two-sided hypothesis test at significance level $\alpha$

• $t^c$ increases when $(1-\alpha)\%$ increases

• We call $(1-\alpha)\%$ the confidence level

• The width of the interval therefore depends on $t^c$ and $se(\hat{\beta}_{1})$

• Larger standard error makes the interval wider

• A higher confidence level, which increases $t^c$, makes the interval wider

## Confidence Interval for $\beta_{1}$

• Recall direct relationship between confidence intervals and hypothesis testing

• Suppose confidence level is $(1-\alpha) \%$

• Any $H_{0}$ in the interval is not rejected at $\alpha \%$ level

• Any $H_{0}$ outside the interval is rejected at $\alpha \%$ level

• Intuition for 95% confidence interval

• In hypothesis test, accept $H_{0}$ if $\hat{\beta}_{1}$ is less than 1.96 standard deviations away from claim

• With confidence interval, we find values 1.96 standard deviations away from $\hat{\beta}_{1}$

• Then, if we set $H_{0}$ in that range, then $\hat{\beta}_{1}$ is less than 1.96 standard deviations away

• If we set $H_{0}$ outside the range, is more than 1.96 standard deviations away

## Confidence Interval for $\beta_{1}$

• Can compute confidence interval for any $\alpha$

• If $\alpha = 0.01$, this is a 99% confidence interval

• If $\alpha = 0.10$, this is a 90% confidence interval

• $(1-\alpha) \%$ confidence interval is related to hypothesis test at $\alpha \%$ level

• Recall that $\beta_{1}$ is effect of one-unit change in $X$ on $Y$

• Can construct more confidence interval for general in $X$

• If $\beta_{1}$ is effect of one-unit change, $\Delta x \beta_{1}$ of a change $\Delta x$

• e.g., Effect of 2-unit change in $X$ is $2\beta_{1}$

## Confidence Interval for $\beta_{1}$

• A confidence interval for $\Delta x \beta_{1}$ is $\Delta x\hat{\beta}_{1} \pm t^c \times SE(\hat{\beta}_{1}) \Delta x$

• or equivalently

$\{ (\hat{\beta}_{1} - t^c \times SE(\hat{\beta}_{1})) \Delta x, (\hat{\beta}_{1} + t^c \times SE(\hat{\beta}_{1})) \Delta x \}$

• Effectively, the interval is scaled by $\Delta x$

• Notice that if you set $\Delta x = 1$, you get the original formula

## Example with Stata

• Recall the research question: Are class size and student achievement related?

• The underlying population regression function is

$TestScore_{i}= \beta_{0} + \beta_{1}STR_{i} + u_{i}$

• $\beta_{1}$ is ceteris paribus effect of one more student per teacher

• We estimated $\beta_{1}$ and $\beta_{0}$ using OLS on simulated data

## Example with Stata

• Command for OLS estimates is “regress”

• The output from that command is summarized below

clear
set obs 420
set seed 12345

gen str = rnormal(20,2)
gen u = rnormal(0,20)

gen testscr = 700 -2 * str + u

regress testscr str
Number of observations (_N) was 0, now 420.

Source |       SS           df       MS      Number of obs   =       420
-------------+----------------------------------   F(1, 418)       =     15.45
Model |  6383.10498         1  6383.10498   Prob > F        =    0.0001
Residual |  172661.265       418  413.065226   R-squared       =    0.0357
Total |  179044.369       419  427.313531   Root MSE        =    20.324

------------------------------------------------------------------------------
testscr | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
str |  -1.855817    .472094    -3.93   0.000    -2.783791   -.9278429
_cons |   696.4934    9.55519    72.89   0.000     677.7112    715.2756
------------------------------------------------------------------------------

## Example with Stata

• Important results for hypothesis testing is in bottom of table

• The “std. err.” column reports the standard errors for the estimates

• $SE(\hat{\beta}_{1}) = .472$
• The “t” column shows the t-statistic for $H_{0}: \beta_{1,0} =0$

• $t^{act} = -3.93$

• Observed $\hat{\beta}_{1}$ is 3.93 standard deviations below zero

• The “P $> |t|$” column shows the 2-sided p-value for $H_{0}: \beta_{1,0} =0$

• p-value is 0.000

• 0.000% of values are more extreme than the one we observe

• We reject $H_{0}$ at most significance levels

## Example with Stata

• The Stata output also reports a 95% confidence interval

• Interval is $\{ -2.78, -0.93\}$

• Any $H_{0}$ between these numbers is accepted at 5% level

• All others are rejected

• Suppose you want to test other hypotheses

• e.g., what if we want to test $H_{0}: \beta_{1,0} =2$?
• For this, we need to use additional commands

• The easiest way to do two-sided tests is the “test” command

• NOT the “ttest” command, which tests hypotheses about population means

## Example with Stata

• Now test whether the coefficient on $str$ equals 2

• Notice it reports F(1,418)

• This is the F-statistic that we will learn later

• Conclusion based on this test is the same as a t-test

• In fact, $F = t^2$

• So, can do test with p-value

test str = -2
 ( 1)  str = -2

F(  1,   418) =    0.09
Prob > F =    0.7602
• You need to use the test command immediately after the regression

# Regression when $X$ is a Binary Variable

## Introduction

• Up to now we have examined only quantitative variables

• Test scores

• Income

• Schooling

• In many applications, we are interested in qualitative factors

• Gender

• Race

• Location

• In this section we discuss how to incorporate qualitative information into a regression

## Categorical Variables

• Qualitative factors are typically categorical in nature

• Gender: {male, female}

• Marital Status: {married, single}

• City: {Toronto, Montreal, Waterloo, ...}

• These variables separate data into distinct groups

• In many cases, the values the variable can take are not numeric

• For variables with 2 categories, we often code them numerically as dummy variables

• Dummy Variable: A binary, 0-1 variable that describes the values of a qualitative variable with two categories

## Dummy Variables

• Examples

• Code gender into a dummy variable called female

• Gender: {male, female} $\rightarrow$ female = {0,1}

• The variable female = 0 if male, and 1 if female

• Code marital status into a dummy variable called married

• Gender: {married, single} $\rightarrow$ married = {0,1}

• The variable married = 0 if single, and 1 if married

• In both cases, we could reverse the coding

• Could instead define male = 0 if female, and 1 if male

• and single = 0 if married, and 1 if single

• Key is the variable name usually indicates event with value 1

## Dummy Variables

• Point of using 0,1 is that it leads to useful interpretations in data analysis and regression models

• Sample mean of 0,1 variable is fraction of values that equal 1

• If female = {0,1}, $\frac{1}{N}\sum_{i=1}^{N} female_{i}$ = fraction female

• If married = {0,1}, $\frac{1}{N}\sum_{i=1}^{N} married_{i}$ = fraction married

• In regression models, parameters on dummy variables also have useful interpretations

• We will learn these details later in this section
• We will focus only on variables with two categories

• Later in the course we may discuss variables with more than two categories

## Interpretation of Regression Coefficients

• The mechanics of OLS are the same with binary $X$ variables

• Still minimize the sum of squared residuals
• It is only the interpretation of $\beta_{1}$ and $\beta_{0}$ that change

• Imagine we want to measure the effect of class size on test scores

• We only have access to a binary variable on class size

$D_{i} = 1\{str_{i} \ge 20 \}$

• $D_{i}$ equals 0 if student teacher ratio is $<20$

• $D_{i}$ equals 1 if student teacher ratio is $\ge 20$

## Interpretation of Regression Coefficients

• The regression model in this context is $Y_{i} = \beta_{0} + \beta_{1}D_{i} + u_{i}$

• The population regression function is $E[Y_{i}|D_{i}] = \beta_{0} + \beta_{1}D_{i}$

• How do we interpret $\beta_{1}$ in this regression?

• To see this, compute the conditional expectation for each value of $D_{i}$ $E[Y_{i}|D_{i}=1] = \beta_{0} + \beta_{1}$ $E[Y_{i}|D_{i}=0] = \beta_{0}$

• Then take the difference between the two $E[Y_{i}|D_{i}=1] - E[Y_{i}|D_{i}=0] = \beta_{1}$

## Interpretation of Regression Coefficients

• From this, $\beta_{1}$ is the difference in the average value of $Y_{i}$ between the two groups

• Average value of $Y_{i}$ for large classes minus average value of $Y_{i}$ for small classes
• The same is true for any dummy variable

• $\beta_{1}$ is average value of $Y_{i}$ when dummy variable equals 1 minus average value of $Y_{i}$ when dummy variable equals zero
• Thus, the interpretation is not a “slope”

• Instead it is a difference in means
• Also notice interpretation of $\beta_{0}$

• Average value of $Y_{i}$ when dummy variable equals 0

## Interpretation of Regression Coefficients

• The above interpretation is for the regression parameter $\beta_{1}$

• The OLS estimator $\hat{\beta}_{1}$ has an analogous interpretation

• Sample average of $Y_{i}$ when dummy variable equals 1, minus sample average of $Y_{i}$ when dummy variable equals zero
• $\hat{\beta}_{0}$ is the sample mean when the dummy variable equals 0

• You can test hypotheses about $\beta_{1}$ using regular t tests

• In this context, you are testing difference in sample means between groups
• Confidence intervals are also constructed in the same way

## Example with Stata

• In example below, we create a dummy variable for class size

• Then compare regression estimate to difference in means

gen d = 1 if str >=20
replace d = 0 if str <20

regress testscr d
(206 missing values generated)

Source |       SS           df       MS      Number of obs   =       420
-------------+----------------------------------   F(1, 418)       =      3.56
Model |  1510.61982         1  1510.61982   Prob > F        =    0.0600
Residual |   177533.75       418  424.721889   R-squared       =    0.0084
Total |  179044.369       419  427.313531   Root MSE        =    20.609

------------------------------------------------------------------------------
testscr | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
d |  -3.793689   2.011576    -1.89   0.060    -7.747755    .1603765
_cons |   661.0675   1.435882   460.39   0.000      658.245    663.8899
------------------------------------------------------------------------------

## Example with Stata

• Compare regression coefficients to differences in means
sum testscr if d == 1
sum testscr if d == 0
    Variable |        Obs        Mean    Std. dev.       Min        Max
-------------+---------------------------------------------------------
testscr |        214    657.2738    20.45359   597.7711   709.1503

Variable |        Obs        Mean    Std. dev.       Min        Max
-------------+---------------------------------------------------------
testscr |        206    661.0675     20.7688    593.118   713.0748

# Heteroskedasticity vs Homoskedasticity

## Definitions

• For OLS estimators, we made assumptions about the average value of $u_{i}$ conditional on $X_{i}$

• More specifically, we assumed $E[u_{i}|X_{i}] = 0$
• For hypothesis testing, we also need to make assumptions about the variance of $u_{i}$ at each $X_{i}$

• Homoskedasticity: the variance of $u_{i}$ conditional on $X_{i}$ is constant

• Mathematically, $VAR[u_{i}|X_{i}] = \sigma_{u}^2$
• Heteroskedasticity: the variance of $u_{i}$ conditional on $X_{i}$ varies across observations

• Mathematically, $VAR[u_{i}|X_{i}] = \sigma_{ui}^2$

• Difference is that $\sigma_{ui}^2$ varies across individuals

## Intuition

• Recall that errors $u_{i}$ are difference between average $E[Y_{i}|X_{i}]$ and actual $Y_{i}$

• Therefore

• Homoskedasticity means $Y_{i}$ values have same spread around mean at each $X_{i}$

• Heteroskedasticity means $Y_{i}$ values may be spread differently around mean at each $X_{i}$

• How the errors are spread out has several important implications that we discuss below

## Implications

• Why does the spread of the errors at each $X_{i}$ matter?

• There are two key implications

1. Under homoskedasticity, OLS estimators are Best Linear Unbiased Estimators (BLUE)

• Compared to all linear unbiased estimators, they have the lowest variance

• Under heteroskedasticity, this is not true

2. Variance of OLS estimators is different

• Under heteroskedasticity, estimated variance of $\hat{\beta}_{1}$ is the formula derived earlier

$\hat{\sigma}^2_{\beta_{1}}=\frac{1}{n}\frac{\frac{1}{n-2} \sum_{i=1}^{n} (X_{i} - \bar{X})^2\hat{u}_{i}^2}{\left[ \frac{1}{n} \sum_{i=1}^{n} (X_{i} - \bar{X})^2 \right ]^2}$

## Implications

1. Variance of OLS estimators continued...

• Under homoskedasticity, estimated variance of $\hat{\beta}_{1}$ simplifies to $\tilde{\sigma}^2_{\beta_{1}}=\frac{\frac{1}{n-2} \sum_{i=1}^{n} \hat{u}_{i}^2}{ \sum_{i=1}^{n} (X_{i} - \bar{X})^2}$

• If you think errors are heteroskedastic you must use the first formula

• Use of homoskedastic formula will lead to incorrect hypothesis testing

• Because you will underestimate actual standard error estimator

• Overestimates the size of the extreme part of distribution of t-statistic

• The critical values will be too low

• Leads to over-rejecting $H_{0}$

• Confidence intervals based on incorrect standard error also wrong

## Implications

1. OLS estimators are still unbiased with Normal distribution

• The distribution of the errors has no impact on expected value of $\hat{\beta}_{1}$

• $\hat{\beta}_{1}$ is unbiased under homoskedasticity or heteroskedasticity

• As long as other three assumptions hold true

• Distribution of $\hat{\beta}_{1}$ also remains Normal in large samples

• For this reason, heteroskedasticity is only a problem for inference

## What to do in Practice?

• How do we know if errors are heteroskedastic or not?

• Answer is most of the time we do not

• For this reason, best to use heteroskedasticity standard errors as default

• Only use homoskedasticity ones in special cases
• Note: as default, Stata produces the homoskedasticity standard errors for regressions

• To get the ones consistent with heteroskedasticity, you must use the robust option in regress

## Example with Stata

• We will generate new data imposing heteroskedasticity on the error

• Code below makes $u$ increasing with $str^4$

• Higher values of $str$ will have more spread in $testscr$

• This is one type of heteroskedasticity

clear
set obs 420
set seed 12345
gen str = rnormal(20,2)
gen u = rnormal(0,.0005*str^4)

gen testscr = 700 -2 * str + u

## Example with Stata

twoway scatter testscr str, title("Data with Heteroskedastic Errors") scheme(s1mono)

## Example with Stata

• Code and results below show OLS with assumption of homoskedastic errors
regress testscr str
      Source |       SS           df       MS      Number of obs   =       420
-------------+----------------------------------   F(1, 418)       =      4.32
Model |  41914.6004         1  41914.6004   Prob > F        =    0.0383
Residual |  4056457.49       418  9704.44375   R-squared       =    0.0102
Total |  4098372.09       419  9781.31764   Root MSE        =    98.511

------------------------------------------------------------------------------
testscr | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
str |  -4.755562   2.288255    -2.08   0.038    -9.253484   -.2576402
_cons |   752.5441   46.31433    16.25   0.000     661.5061    843.5821
------------------------------------------------------------------------------

## Example with Stata

• Code and results below show OLS with assumption of heteroskedastic errors

• The standard error has become larger

regress testscr str, robust
Linear regression                               Number of obs     =        420
F(1, 418)         =       1.87
Prob > F          =     0.1725
R-squared         =     0.0102
Root MSE          =     98.511

------------------------------------------------------------------------------
|               Robust
testscr | Coefficient  std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
str |  -4.755562   3.480204    -1.37   0.173    -11.59644     2.08532
_cons |   752.5441   67.19808    11.20   0.000     620.4559    884.6324
------------------------------------------------------------------------------

## Example with Stata

• Below we simulate a t-test with heteroskedastic and homoskedastic errors

• Show what happens to t-distribution when null hypothesis is true

• We use $\beta_{1} = 0$ as the null hypothesis

## Example with Stata

clear
set more off
set obs 999
set seed 12345

gen t_he = .
gen t_ho = .

foreach iter of numlist 1/999 {

preserve
clear
qui set obs 420

gen str = rnormal(20,2)
gen u = rnormal(0,.0005*str^4)

gen testscr = 700  + u

qui regress testscr str
local t_ho = _b[str]/_se[str]

qui regress testscr str, robust
local t_he = _b[str]/_se[str]

restore

replace t_ho = t_ho' in iter'
replace t_he = t_he' in iter'

}

## Example with Stata

twoway (kdensity t_ho) (kdensity t_he) (function tden(420-2,x), range(-5 5)), xtitle(t) title(t-distribution when Null Hypothesis is True) legend(order(1 "Homoskedastic Errors" 2 "Heteroskedastic Errors" 3 "Actual t-dist")) scheme(s1color)

## Example with Stata

• Below is the fraction of t values we reject using the critical values from t-distribution

• We chose critical value so that $\alpha = 0.05$

• So we should reject 5% of the time
• With homoskedasticity standard errors we reject 10% of the time

• We “over-reject” when the null hypothesis is true
• With robust standard errors, it is closer to 5%

• The robust errors generate better hypothesis testing

• You should use them all the time
gen reject_ho = abs(t_ho) >=invttail(420-2,0.025)
gen reject_he = abs(t_he) >=invttail(420-2,0.025)

tab1 reject*
-> tabulation of reject_ho

reject_ho |      Freq.     Percent        Cum.
------------+-----------------------------------
0 |        899       89.99       89.99
1 |        100       10.01      100.00
------------+-----------------------------------
Total |        999      100.00

-> tabulation of reject_he

reject_he |      Freq.     Percent        Cum.
------------+-----------------------------------
0 |        951       95.20       95.20
1 |         48        4.80      100.00
------------+-----------------------------------
Total |        999      100.00

## The Gauss-Markov Theorem

• OLS estimators are the most widely-used to estimate regression paramters

• The reason is based on the Gauss-Markov theorem

• Gauss-Markov Theorem: Under the Gauss-Markov conditions, OLS is the Best Linear Unbiased Estimator for $\beta_{1}$

• The Gauss-Markov conditions are the assumptions we discussed before

• Zero conditional mean of the errors

• $X_{i}, Y_{i}$ are iid

• Large outliers are unlikely

• Homoskedastic errors

• If all these hold, OLS estimators have smallest variance among all linear unbiased estimators

• This means, they have the lowest amount of sampling variation