Time Series Regression

Henry Thompson

email for information

Introduction

Suppose an economic model reduces to the effect of exogenous variable X_t on endogenous variable Y_t with exogenous control variable Z_t. Subscripts refer to the time period. In general functional form, the economic relationship of interest is

Y_t = f(X_t, Z_t) (1)

where X_t and Z_t may be vectors. This introduction to time series regression focuses on estimating regressions based on (1). The issue for economic theory is the significance, sign, and size of the partial derivative effect of X_t on Y_t holding Z_t constant. Lags X_t-1 and Z_t-1 may be more critical than contemporaneous effects, extending to X_t-i and Z_t-j in high frequency data.

The reduced form equation (1) should be derived from a structural theoretical model. In time series “reduced form” also refers to a model with lags of dependent variable that are exogenous in the sense they occurred previously, Y_t = f(X_t-i, Z_t-j). Estimated parameters in (1) can lead to the derivation of structural coefficients in the theoretical model. For instance the demand elasticity in a market model can be derived from the estimated structural equations in the market model.

Economic time series depend in part on the process underlying their history. The best predictor of y_t+1 may be y_t. The y_t series reflects the history of the influence of truly related variables under actual circumstances. Univariate regressions of y_t involving only its own history plus perhaps time itself as a variable can be very successful predictors of y_t+1. Such univariate models do not, however, reveal economic relationships of interest since economics is based on relationships between variables. Univariate pretests are critical for economic analysis, however, as properties of the series in (1) lay the foundation for successful specification of regressions.

Models with lags of independent variables are better predictors than univariate models when unexpected changes occur in independent variables. Successful estimates of reduced form equations isolate the effects of exogenous variables and may lead to derived structural parameters. The goals of time series regression are to focus and improve economic theory.

OLS regression assumes normally distributed variables with each observation equal to its mean plus a random error term. A positive regression coefficient indicates above average observations of X_t are associated with above average observations of Y_t holding exogenous control variable Z_t constant. Standard errors are based on variables with normal distributions.

If a series has a positive trend, early observations are below the mean and later ones above it. There is no clustering around the mean as with normal distributions, the series simply passing through the mean. A trending variable has a low peak and fat tails relative to the normal distribution. A regression on nonstationary variables understates standard errors resulting in inflated parameter significance and overstated explanatory power. The first step to estimate (1) is to identify the underlying autoregressive processes to ensure the lack of trends in every variable. If (1) is estimated with trending variables, the standard errors will be understated.

The typical problem in applied time series regression is trending variables. If theory suggests X_t should have a positive effect on Y_t and both have positive trends, they will be correlated and regression coefficients will appear significant. Observations of X_t below its mean will be associated with observations of Y_t below its mean simply because they occur at an earlier time. The resulting residual correlation overstates parameter significance and explanatory power. Residual correlation implies information remains in the residual, suggesting model misspecification.

A stationary time series converges to a dynamic equilibrium steady state. The series may not be normally distributed as it approaches the steady state but should be nearly so. Sample periods do not include all of variable “history” assumed to extend into the future as well. The sample selection period may result in a trending variable that could be stationary or even normal with a longer sample. If variables in a regression are stationary, standard errors are typically reliable. Applied time series analysis focuses in some part on these distorted standard errors due to trends.

A simple regression related to (1) is

y_t = α₀ + α₁x_t + α₂z_t + ε_t. (2)

Variables are transformed to natural logs, y_t = lnY_t and so on. Log linear regression coefficients are point estimates of elasticities. The goal is to interpret theory in terms of the estimated elasticity α₁ = ∂y_t/∂x_t although the ultimate form of the time series regression may not be as simple as (2). The individual time series processes determine the form of variables in (2). Variables can be transformed with differences and lags. The regression may include structural breaks, time itself, and variance of the series. Variables in (2) should be stationary. White noise residuals of underlying univariate processes are candidate variables for (2). The error correction model ECM includes the residual ε_t of (2) in a difference equation regression.

Begin with a theoretical model deriving (2) to relate theoretical parameters to estimated coefficients. Rely on theory and preliminary regressions to suggest the most relevant exogenous and control variables. The residual ε_t has to be white noise WN with zero mean, lack of residual correlation, and constant variance. The residual correlation r(ε_t, ε_t-1) from (2) plays a critical role in time series regression.

Even in the presence of residual correlation, the estimated coefficient a₁ in (2) is unbiased or just as likely above as below its true value. It is consistent, converging to its true value as the number of observations increases. It is also super consistent with accelerating convergence as the number of observations increases. In the presence of residual correlation, however, it is impossible to say a₁ is not zero due to the understated standard errors.

Spurious regressions occur when unrelated trending variables appear related in regressions due to the underestimated standard errors. Arbitrary choice of trending variables can result in apparently successful regressions. To avoid spurious regressions, rely on economic theory to select variables. Relying on economic theory, spurious regressions are not an issue.

If the series in a regression are not stationary but their differences are, regressions on differences produce more reliable estimates. Difference stationary series may also be cointegrated suggesting an error correction model that includes transitory adjustment relative to the long term dynamic equilibrium. Unsuccessful difference regressions may disguise a significant error correction processes. Partial adjustment models introduce the lag of the dependent variable as an exogenous variable.

In an economic model with more than a single endogenous dependent variable, solve the reduced form equations with each dependent variable a function of exogenous variables. Estimate each of the reduced form equations. For example, the market model determines endogenous price P and quantity Q from the demand function D = D(P, Y), the supply function S = S(P, W), and the equilibrium condition Q = S = D. Exogenous variables are the demand shifter Y and the supply shifter W. Estimate P or Q as functions of Y and W, but not P as a function of Q or vice versa.

The macroeconomic model provides another example. National income Y is a function of exogenous government spending G, money supply M, the foreign interest rate r*, and foreign income Y*. The interest rate r is endogenous and should not be a right hand variable in a regression with endogenous Y, or vice versa. A floating exchange rate E would be endogenous since it adjusts to a nonzero trade balance. With a fixed exchange rate E would be exogenous and B endogenous.

Theory is flexible in that various assumptions lead to different reduced form equations. For instance, price is exogenous in international economics by the small open economy assumption. Time series evidence provides tests of particular assumptions as theory stands ready to work through the implications of alternative assumptions.