Probabilistic foundations of Autoregressive AR(1) Models.
Autoregressive time series models are the fundamental mathematical tool for analyzing sequential, temporally correlated observations in econometrics, meteorology, signal processing, and quantitative finance. Grounded in stochastic difference equations and the Yule-Walker recurrence relations, the first-order AR(1) model captures memory and persistence while maintaining weak stationarity under strict unit-circle parameter constraints. In this laboratory, we simulate autoregressive realizations, compute empirical autocorrelation (ACF) and partial autocorrelation (PACF) signatures, test for unit roots, and confirm the parameter scaling gradient m = 0.511. In mathematical statistics, empirical data science, and modern computational engineering, understanding the asymptotic sampling properties and theoretical behavior of Autoregressive AR(1) Time Series, Yule-Walker Equations, and Weak Stationarity provides crucial guarantees. Rigorous experimental validation connects foundational probability laws with observable empirical phenomena, ensuring that statistical inference remains valid under finite sample sizes and real-world data collection constraints.
Viva voce questions & analytical theory
How is the first-order Autoregressive AR(1) time series model defined mathematically?
A discrete-time stochastic process {X_t, t ∈ ℤ} follows an autoregressive model of order 1, denoted AR(1), if it satisfies the stochastic difference equation: X_t = c + φ X_{t-1} + ε_t, where c is a constant drift parameter, φ is the autoregressive persistence parameter, and {ε_t} is a white noise process satisfying E[ε_t] = 0, Var(ε_t) = σ_ε^2, and Cov(ε_t, ε_s) = 0 for all t ≠ s. The current value X_t is modeled as a linear combination of its immediately preceding value X_{t-1} plus an unforecastable contemporary shock ε_t.
What is weak (covariance) stationarity, and what condition must φ satisfy in an AR(1) model?
A time series {X_t} is weakly stationary (covariance stationary or wide-sense stationary) if: (1) Its mean is constant and time-invariant: E[X_t] = μ for all t; (2) Its autocovariance Cov(X_t, X_{t-k}) = γ_k depends solely on the time separation (lag) k, and not on absolute time t; and (3) Its variance γ_0 = Var(X_t) < ∞ is finite. For an AR(1) process, taking expectations gives μ = c + φ μ ⟹ μ = c / (1 - φ). By recursive back-substitution X_t = ∑_{j=0}^∞ φ^j ε_{t-j}, the sum converges in mean square if and only if |φ| < 1. If |φ| = 1, the process is a non-stationary random walk with a unit root.
How are the Yule-Walker equations derived for the AR(1) autocorrelation function?
Subtracting mean μ from both sides yields centered form x̃_t = φ x̃_{t-1} + ε_t. Multiplying both sides by x̃_{t-k} (for k ≥ 1) and taking mathematical expectations: E[x̃_t x̃_{t-k}] = φ E[x̃_{t-1} x̃_{t-k}] + E[ε_t x̃_{t-k}]. Because future noise ε_t is uncorrelated with past values x̃_{t-k}, E[ε_t x̃_{t-k}] = 0. This yields the autocovariance recurrence γ_k = φ γ_{k-1}. Dividing by variance γ_0 produces the Yule-Walker autocorrelation recurrence: ρ_k = φ ρ_{k-1}. Since ρ_0 = 1, by mathematical induction: ρ_k = φ^k for all k ≥ 0, establishing geometric exponential decay.
What is the theoretical variance of a stationary AR(1) process?
Taking the variance of both sides of X_t = φ X_{t-1} + ε_t: Var(X_t) = Var(φ X_{t-1} + ε_t). By stationarity, Var(X_t) = Var(X_{t-1}) = γ_0. Because ε_t is independent of X_{t-1}, the variance of the sum is the sum of variances: γ_0 = φ^2 γ_0 + σ_ε^2. Rearranging gives γ_0 (1 - φ^2) = σ_ε^2, which yields the stationary variance formula: γ_0 = σ_ε^2 / (1 - φ^2). As |φ| approaches 1, the stationary variance diverges toward infinity, demonstrating the onset of non-stationary unit root behavior.
How does the Partial Autocorrelation Function (PACF) distinguish AR models from MA models?
The Partial Autocorrelation Function (PACF) at lag k, denoted α_k or φ_{kk}, measures the correlation between X_t and X_{t-k} after removing the linear effects of all intermediate lags X_{t-1}, ..., X_{t-k+1}. For an AR(p) process: (1) The Autocorrelation Function (ACF) decays exponentially or as damped sinusoids without cutting off; (2) The Partial Autocorrelation Function (PACF) cuts off sharply to zero after lag p (φ_{kk} = 0 for all k > p). For AR(1), PACF exhibits a single significant spike at lag 1 (φ_{11} = φ) and is zero thereafter, providing the primary diagnostic signature for model identification.
What is the Dickey-Fuller unit root test, and why do standard t-distributions fail?
The Dickey-Fuller test evaluates the null hypothesis of a non-stationary unit root H_0: φ = 1 (or ΔX_t = (φ - 1)X_{t-1} + ε_t ⟹ γ = 0 where γ = φ - 1). Under H_0, the process is a random walk; the denominator of the OLS t-statistic converges to a stochastic integral involving Brownian motion rather than a Gaussian variable. Consequently, the test statistic t = γ̂ / SE(γ̂) does not follow Student's t-distribution, but rather the skewed Dickey-Fuller distribution (tabulated via Monte Carlo simulation). Failing to reject H_0 warns of spurious regression.
What is Spurious Regression in non-stationary time series?
Clive Granger and Paul Newbold (1974) demonstrated that running Ordinary Least Squares regression between two independent, completely unrelated non-stationary random walks Y_t and X_t yields high R² values and statistically significant t-ratios over 75% of the time. This 'spurious regression' occurs because non-stationary series wander together in long-term trends, generating false correlation. Differencing the series to achieve stationarity (ΔY_t, ΔX_t) or testing for cointegration resolves this hazard.
Experimental precautions and statistical checks
- Ensure the autoregressive persistence parameter strictly satisfies |φ| < 1 to guarantee stationarity.
- Discard an initial burn-in sequence (at least 100 observations) to eliminate starting-state transient effects.
- Verify that residuals from fitted AR models exhibit zero autocorrelation (Gaussian white noise checks).
- Perform unit-root testing (Augmented Dickey-Fuller) before fitting stationary autoregressive models to real data.
Theoretical assumptions & limitations
- AR(1) models assume linear temporal dependencies; non-linear dynamic patterns require threshold autoregressive (TAR) models.
- Structural breaks or regime shifts in the mean or persistence parameter can mimic unit root non-stationarity.