Asymptotics & Heteroskedasticity

• But normality is a strong assumption. What if the errors are not normal?
• Large-sample (asymptotic) theory provides approximate distributions under weaker assumptions, valid when \(n\) is large.

We need two key tools: the Law of Large Numbers and the Central Limit Theorem.

Notation: \(\theta_n \xrightarrow{p} \theta\) or \(\text{plim}\, \theta_n = \theta\).

Informally: as \(n\) grows, \(\theta_n\) is increasingly likely to be close to \(\theta\).

• The sample average converges to the population mean.
• More generally, any sample moment converges to its population counterpart: \(\frac{1}{n}\sum_{i=1}^n g(X_i) \xrightarrow{p} E[g(X_i)]\).

Continuous mapping: if \(g(\cdot)\) is continuous at \(\theta\), then \(g(\theta_n) \xrightarrow{p} g(\theta)\).

• As \(n\) increases, the estimator concentrates around the true value.
• Consistency is a minimum requirement for sensible estimators.

“If you can’t get it right as \(n\) goes to infinity, you shouldn’t be in this business.” — C. W. J. Granger

Note: we do not need homoskedasticity (MLR.5) or normality (MLR.6).

Denominator: \(\frac{1}{n}\sum(x_i - \bar{x})^2 = \frac{1}{n}\sum x_i^2 - \bar{x}^2 \xrightarrow{p} E[x_i^2] - (E[x_i])^2 = \text{Var}(x_i)\)

Numerator: expand as \(\frac{1}{n}\sum x_i u_i - \bar{x}\cdot\frac{1}{n}\sum u_i\).

• \(\frac{1}{n}\sum x_i u_i \xrightarrow{p} E[x_i u_i] = 0\)   (LLN + MLR.4)
• \(\bar{x} \xrightarrow{p} E[x_i]\),   \(\frac{1}{n}\sum u_i \xrightarrow{p} 0\)
• By Slutsky: numerator \(\xrightarrow{p} 0\).

• MLR.4: \(E(u \mid x_1, \ldots, x_k) = 0\)   (zero conditional mean)
• MLR.4’: \(E(u) = 0\) and \(\text{Cov}(x_j, u) = 0\) for all \(j\)   (zero mean and zero correlation)

• MLR.4 implies MLR.4’, but not vice versa.
• MLR.4 \(\Rightarrow\) unbiasedness (finite-sample property).
• MLR.4’ \(\Rightarrow\) consistency (large-sample property).

• \(v = \beta_2 x^2 + u - \beta_2 E(x^2)\), so \(E(v \mid x) = \beta_2(x^2 - 1) \neq 0\).
• MLR.4 is violated: \(\hat\beta_1\) is biased.

• But \(\text{Cov}(x, v) = E[xv] = \beta_2 E[x^3] = 0\) (odd moments of a symmetric distribution vanish).
• MLR.4’ holds: \(\hat\beta_1\) is consistent.

• The second term is the asymptotic bias.
• More data does not help: as \(n \to \infty\), \(\hat\beta_1\) converges to the wrong value.

\[ \text{plim}\, \tilde\beta_1 = \beta_1 + \beta_2 \frac{\text{Cov}(x_1, x_2)}{\text{Var}(x_1)} \]

Same structure as finite-sample OVB, but now stated as a probability limit.

• This is convergence of CDFs, not of the random variables themselves.
• Weaker than convergence in probability.

• The standardized sample average is approximately normal in large samples, regardless of the population distribution.
• This is the foundation for asymptotic inference.

If \(Z_n \xrightarrow{d} N(0,1)\), then \(Z_n^2 \xrightarrow{d} \chi^2_1\).

These results let us combine convergence in probability (for consistent estimators) with convergence in distribution (from the CLT).

• No normality assumption (MLR.6) needed.
• The CLT drives the normal approximation.

Denominator: \(\xrightarrow{p} \text{Var}(x_i)\)   (as in the consistency proof)

Numerator: expand as \(\frac{1}{\sqrt{n}}\sum(x_i - E[x_i])u_i + (E[x_i] - \bar{x})\frac{1}{\sqrt{n}}\sum u_i\).

• First term: \((x_i - E[x_i])u_i\) are i.i.d. with mean zero, so by CLT \(\xrightarrow{d} N(0, E[(x_i - E[x_i])^2 u_i^2])\).
• Second term \(\xrightarrow{p} 0\).

Combining by Slutsky’s theorem:

\[ \sqrt{n}(\hat\beta_1 - \beta_1) \xrightarrow{d} N\!\left(0, \; \frac{\sigma^2}{\text{Var}(x_i)}\right) \]

General result (MLR.1–MLR.5):

\[ \hat\beta_j \overset{a}{\sim} N\!\left(\beta_j, \; \frac{\sigma^2}{\text{SST}_j(1 - R_j^2)}\right) \]

where \(\text{SST}_j = \sum(x_{ij} - \bar{x}_j)^2\) and \(R_j^2\) is the \(R^2\) from regressing \(x_j\) on the other regressors. The denominator \(\text{SST}_j(1 - R_j^2) = \sum \hat{r}_{ij}^2\) is the residual variation in \(x_j\).

• Reject \(H_0\) at level \(\alpha\) if \(|t| > z_{1-\alpha/2}\).
• Similarly, \(F_{q,\, n-k-1} \to \chi^2_q / q\) as \(n \to \infty\).
• In practice, software uses \(t_{n-k-1}\) and \(F_{q,\, n-k-1}\), which are more conservative and coincide with the exact results under MLR.6.

The standard error \(\text{se}(\hat\beta_j)\) is called the asymptotic standard error when MLR.6 is not assumed.

• In large samples, the same \(t\)- and \(F\)-tests are valid either way.
• The distinction matters most in small samples where the normal approximation may be poor.

This is the large-sample analog of the Gauss-Markov theorem (BLUE).

Large-sample (MLR.1–MLR.5):

• Consistency (MLR.1–4)
• Asymptotic normality, approximate \(t\) and \(F\) (MLR.1–5)
• Asymptotic efficiency (MLR.1–5)

The key tradeoff: we drop normality (MLR.6), but results are now approximations that require large \(n\).

• The error variance is the same for all values of \(X\).
• This gave us the variance formula \(\text{Var}(\hat\beta_j \mid X) = \sigma^2 / [\text{SST}_j(1 - R_j^2)]\).

What if MLR.5 fails?

Still valid:

• OLS is unbiased and consistent
• OLS coefficients are asymptotically normal (but with a different variance)

No longer valid:

• The usual variance formula \(\hat\sigma^2 / [\text{SST}_j(1 - R_j^2)]\)
• The usual \(t\)- and \(F\)-statistics (which rely on that formula)
• OLS is no longer BLUE (or asymptotically efficient)

Under homoskedasticity, \(E[u_i^2 \mid x_i] = \sigma^2\), so we can simplify:

\[ \begin{aligned} E[(x_i - E[x_i])^2 u_i^2] &= E\!\big[(x_i - E[x_i])^2\, E[u_i^2 \mid x_i]\big] \\ &= \sigma^2\, E[(x_i - E[x_i])^2] = \sigma^2 \text{Var}(x_i) \end{aligned} \]

and \(V_1 = \sigma^2 / \text{Var}(x_i)\).

Under heteroskedasticity, \(E[u_i^2 \mid x_i]\) varies with \(x_i\), so \(\sigma^2\) cannot be pulled out. The usual formula understates or overstates the true variance, leading to invalid inference.

• Each squared residual \(\hat{u}_i^2\) gets its own weight, rather than using a common \(\hat\sigma^2\).
• In MLR, the same principle applies: the general form is a “sandwich” estimator. Software computes this automatically.

The resulting standard errors are called heteroskedasticity-robust (or HC) standard errors (White, 1980).

• Under \(H_0\), \(t \xrightarrow{d} N(0,1)\).
• Robust \(F\)-statistics for joint hypotheses are also available.

Caveat: robust standard errors rely on large-sample theory. In small samples, they can be unreliable.

• The differences are small here.
• With strong heteroskedasticity, differences can be substantial.
• To be safe: always report robust standard errors.

Heteroskedasticity is the norm in cross-sectional data, not the exception.