From the free course Econometric Theory and Empirical Analysis with Stata on Lykke

Functional Forms and Heteroskedasticity

Key concepts: Log-Log Models · Quadratic Specifications · Heteroskedasticity · Robust Standard Errors · Breusch-Pagan Test

Explores non-linear relationships using logs and quadratics, and addresses violations of the constant variance assumption.

Functional Forms and Heteroskedasticity

In the pursuit of modeling economic phenomena, we often find that the rigid structure of a simple linear relationship—where a one-unit change in $X$ results in a constant $\beta$ change in $Y$—is insufficient. The real world is characterized by diminishing returns, percentage-based growth, and variances that fluctuate alongside the scale of the data. To capture these nuances, the econometrician must master two critical domains: the selection of Functional Forms to capture non-linearities and the treatment of Heteroskedasticity to ensure valid statistical inference.

Non-Linear Functional Forms

While the Ordinary Least Squares (OLS) estimator requires the model to be linear in parameters, it does not require the model to be linear in variables. This distinction is the gateway to modeling complex economic behavior. By transforming variables using logarithms or polynomials, we can model non-linear relationships while remaining within the powerful framework of OLS.

Logarithmic Transformations

Logarithmic models are perhaps the most common non-linear transformations in economics. They are particularly useful when dealing with variables that are strictly positive and have skewed distributions, such as wages, firm size, or population.

The Logarithmic Rule of Thumb: When a variable is logged, we move from discussing "unit changes" to "percentage changes."

There are three primary logarithmic specifications, each with a distinct interpretation of the slope coefficient $\beta_1$:

Model Type Specification Interpretation of $\beta_1$ Name of $\beta_1$
Level-Level $y = \beta_0 + \beta_1 x + u$ $\Delta y = \beta_1 \Delta x$ Slope
Level-Log $y = \beta_0 + \beta_1 \ln(x) + u$ $\Delta y \approx (\beta_1 / 100) % \Delta x$ Semi-elasticity
Log-Level $\ln(y) = \beta_0 + \beta_1 x + u$ $% \Delta y \approx (100 \cdot \beta_1) \Delta x$ Semi-elasticity
Log-Log $\ln(y) = \beta_0 + \beta_1 \ln(x) + u$ $% \Delta y = \beta_1 % \Delta x$ Elasticity

The Log-Log Model (Constant Elasticity)

In a Log-Log Model, both the dependent and independent variables are transformed by the natural logarithm. This is the standard specification for estimating demand functions or production functions (like the Cobb-Douglas model).

Mathematical Derivation of Elasticity: Consider the model: $\ln(y) = \beta_0 + \beta_1 \ln(x) + u$. To find the effect of $x$ on $y$, we take the derivative with respect to $x$: $$\frac{d(\ln y)}{dx} = \beta_1 \frac{d(\ln x)}{dx}$$ Using the chain rule ($\frac{d \ln(y)}{dy} = \frac{1}{y}$): $$\frac{1}{y} \cdot \frac{dy}{dx} = \beta_1 \cdot \frac{1}{x}$$ Rearranging for $\beta_1$: $$\beta_1 = \frac{dy/y}{dx/x} = \frac{% \Delta y}{% \Delta x}$$ Thus, $\beta_1$ represents the elasticity of $y$ with respect to $x$.

import numpy as np
import pandas as pd
import statsmodels.api as sm

# Example: Estimating Price Elasticity of Demand
# Generating synthetic data for Quantity (Q) and Price (P)
np.random.seed(42)
price = np.random.uniform(10, 50, 100)
# Demand function: Q = A * P^(-1.5) -> ln(Q) = ln(A) - 1.5*ln(P)
log_q = 10 - 1.5 * np.log(price) + np.random.normal(0, 0.1, 100)

df = pd.DataFrame({'log_price': np.log(price), 'log_quantity': log_q})

# OLS Regression on Log-Log specification
X = sm.add_constant(df['log_price'])
model = sm.OLS(df['log_quantity'], X).fit()

print(f"Estimated Elasticity: {model.params['log_price']:.4f}")
# Output will be close to -1.5

Quadratic Specifications

Quadratic models are used to capture relationships that have a "turning point," representing either diminishing or increasing marginal effects. A classic example is the relationship between Experience and Wages. Initially, an extra year of experience increases wages significantly, but this effect eventually tapers off or even turns negative as skills become obsolete or retirement nears.

The general form is: $$y = \beta_0 + \beta_1 x + \beta_2 x^2 + u$$

To find the marginal effect of $x$ on $y$, we take the partial derivative: $$\frac{\partial y}{\partial x} = \beta_1 + 2\beta_2 x$$

Insight: The effect of $x$ on $y$ is no longer constant; it depends on the starting value of $x$.

Finding the Turning Point: The "peak" or "trough" of the curve occurs where the marginal effect is zero: $$\beta_1 + 2\beta_2 x^* = 0 \implies x^* = -\frac{\beta_1}{2\beta_2}$$

Heteroskedasticity: Theory and Consequences

In the standard Gauss-Markov framework, we assume Homoskedasticity (Assumption MLR.5): the variance of the error term $u$, conditional on the explanatory variables, is constant. $$Var(u | x_1, x_2, ..., x_k) = \sigma^2$$

Heteroskedasticity occurs when this assumption is violated, meaning the variance of the unobservables changes across different levels of the independent variables: $$Var(u | \mathbf{x}) = h(\mathbf{x})$$

Why Heteroskedasticity Matters

It is a common misconception that heteroskedasticity biases the OLS coefficients. This is false.

Property Impact of Heteroskedasticity
Unbiasedness No Impact. OLS remains unbiased (MLR.1 - MLR.4 still hold).
Consistency No Impact. OLS remains consistent as $n \to \infty$.
Efficiency Lost. OLS is no longer the Best Linear Unbiased Estimator (BLUE).
Standard Errors Invalid. The usual OLS standard error formulas are biased.
Inference Invalid. $t$-statistics and $F$-statistics no longer follow the $t$ and $F$ distributions.

Detecting Heteroskedasticity

Before correcting for heteroskedasticity, we must confirm its presence. While visual inspection of residual plots is a good first step, formal statistical tests provide more rigor.

The Breusch-Pagan (BP) Test

The BP test checks if the variance of the error depends on the explanatory variables in a linear way.

  1. Estimate the OLS model: $y = \beta_0 + \beta_1 x_1 + ... + \beta_k x_k + u$.
  2. Obtain the squared residuals: $\hat{u}^2$.
  3. Run the auxiliary regression: $\hat{u}^2 = \delta_0 + \delta_1 x_1 + ... + \delta_k x_k + v$.
  4. Test the Null Hypothesis: $H_0: \delta_1 = \delta_2 = ... = \delta_k = 0$.
  5. Compute the statistic: $LM = n \cdot R^2_{\hat{u}^2}$, which follows a $\chi^2_k$ distribution.

The White Test

The White test is more general than the BP test because it allows for non-linearities in the variance (by including squares and cross-products of the regressors). However, it consumes many degrees of freedom if there are many $X$ variables.

\text{White Test Auxiliary Regression:} \\
\hat{u}^2 = \delta_0 + \delta_1 x_1 + \delta_2 x_2 + \delta_3 x_1^2 + \delta_4 x_2^2 + \delta_5 (x_1 x_2) + v

Correcting for Heteroskedasticity

Once heteroskedasticity is detected, we have two primary paths: Robust Inference (adjusting the standard errors) or Weighted Least Squares (transforming the model).

Heteroskedasticity-Robust Standard Errors

The most common modern approach is to use Robust Standard Errors (also known as White, Huber, or Eicker standard errors). This method allows us to keep the OLS estimates but adjusts the standard errors to be valid even when the variance is non-constant.

The robust variance estimator for $\hat{\beta}_j$ in a simple regression is: $$\widehat{Var}(\hat{\beta}_1) = \frac{\sum (x_i - \bar{x})^2 \hat{u}_i^2}{SST_x^2}$$

In matrix notation, this is often called the Sandwich Estimator: $$\hat{V}_{robust} = (X'X)^{-1} (X' \hat{\Omega} X) (X'X)^{-1}$$ where $\hat{\Omega}$ is a diagonal matrix of squared residuals.

Weighted Least Squares (WLS)

If we know the exact form of the heteroskedasticity (i.e., $Var(u|x) = \sigma^2 h(x)$), WLS is more efficient than OLS. We "weight" the observations by $1/\sqrt{h(x)}$, effectively giving more importance to observations with lower variance.

Steps for WLS:

  1. Assume $Var(u|x) = \sigma^2 x_i$.
  2. Divide the entire regression equation by $\sqrt{x_i}$.
  3. The transformed error term will now be homoskedastic.

Implementation and Real-World Usage

In professional practice, researchers rarely rely on the assumption of homoskedasticity. The "gold standard" in applied econometrics is to report robust standard errors by default.

// Stata Example: Wage Equation with Functional Forms and Robust SEs

// 1. Load dataset
use "http://www.stata.com/data/jmulti/wage1.dta", clear

// 2. Estimate a Log-Level model with a Quadratic term for Experience
// Model: log(wage) = b0 + b1*educ + b2*exper + b3*exper^2 + u
gen exper_sq = exper^2
reg lwage educ exper exper_sq, robust

// 3. Post-estimation: Find the turning point for experience
// Turning point = -b[exper] / (2 * b[exper_sq])
scalar tp = -_b[exper] / (2 * _b[exper_sq])
display "The wage-maximizing years of experience is: " tp

// 4. Test for Heteroskedasticity (Breusch-Pagan)
// Note: estat hettest usually follows a standard reg (non-robust)
reg lwage educ exper exper_sq
estat hettest

Common Pitfalls and Misconceptions

  1. Logging Zero or Negative Values: The natural log is only defined for $x > 0$. If your data contains zeros (e.g., years of education), researchers often use $\ln(x + 1)$, though this can slightly alter interpretation.
  2. Interpreting Log-Level as Percentages: Remember that for a coefficient $\beta = 0.10$ in a log-level model, the effect is a $10%$ increase, not a $0.10%$ increase. However, for large $\beta$, the approximation $% \Delta y \approx 100 \cdot \beta$ breaks down, and one should use $100 \cdot (e^\beta - 1)$.
  3. Robust SEs are not a Panacea: While robust standard errors fix inference for heteroskedasticity, they do not fix bias caused by omitted variables, measurement error, or endogeneity.
  4. The "Robust" Trade-off: In very small samples, robust standard errors can be less reliable than standard OLS errors if the homoskedasticity assumption actually holds.
Functional Forms and Heteroskedasticity - Econometric Theory and Empirical Analysis with Stata - image 1
Functional Forms and Heteroskedasticity - Econometric Theory and Empirical Analysis with Stata - image 1
Functional Forms and Heteroskedasticity - Econometric Theory and Empirical Analysis with Stata - diagram 1
Functional Forms and Heteroskedasticity - Econometric Theory and Empirical Analysis with Stata - diagram 1
Functional Forms and Heteroskedasticity - Econometric Theory and Empirical Analysis with Stata - diagram 2
Functional Forms and Heteroskedasticity - Econometric Theory and Empirical Analysis with Stata - diagram 2

Source Materials

Study Econometric Theory and Empirical Analysis with Stata on Lykke

This course page is free to open: signed out, you can read its first two sections, and signing in, on any plan, opens the rest. The free Lykke Chrome extension syncs your own Canvas courses, so ChatGPT, Claude and Muse can answer from your classes and deadlines. It only reads Canvas.

Add the free extensionSign in to open every section

All sections of Econometric Theory and Empirical Analysis with Stata · Browse more course pages

Functional Forms and Heteroskedasticity — Econometric Theory and Empirical Analysis with Stata | Lykke