Probabilistic foundations of Pearson vs Spearman Correlation.
Bivariate correlation analysis quantifies the strength and direction of association between two continuous random variables. Pearson's product-moment correlation coefficient evaluates linear dependence, but is vulnerable to outliers and non-linear distortion. In contrast, Spearman's rank correlation evaluates monotonic co-movement by operating on rank-transformed observations, providing a robust non-parametric measure of association. In this laboratory, we generate bivariate normal distributions across varying correlation levels, apply non-linear monotonic warps, test outlier resilience, and verify the linear concordance relationship m = 1.000. In mathematical statistics, empirical data science, and modern computational engineering, understanding the asymptotic sampling properties and theoretical behavior of Pearson Linear Correlation vs Spearman Rank Order Correlation provides crucial guarantees. Rigorous experimental validation connects foundational probability laws with observable empirical phenomena, ensuring that statistical inference remains valid under finite sample sizes and real-world data collection constraints.
Viva voce questions & analytical theory
How is Pearson's product-moment correlation coefficient formulated mathematically?
Karl Pearson (1895) defined the population correlation coefficient as ρ_{X,Y} = Cov(X, Y) / (σ_X σ_Y) = E[(X - μ_X)(Y - μ_Y)] / (σ_X σ_Y). For a sample of paired observations (X_1, Y_1), ..., (X_n, Y_n), the sample correlation coefficient r is given by r = ∑_{i=1}^n (X_i - x̄)(Y_i - ȳ) / [√∑ (X_i - x̄)^2 √∑ (Y_i - ȳ)^2] = S_{xy} / √(S_{xx} S_{yy}). By the Cauchy-Schwarz inequality, -1 ≤ r ≤ +1. A value of r = +1 indicates a perfect positive linear relationship, r = -1 indicates a perfect negative linear relationship, and r = 0 indicates absence of linear correlation.
How is Spearman's rank correlation coefficient formulated, and what does it measure?
Charles Spearman (1904) defined rank correlation as the Pearson correlation computed on the ranks of the data. Let R(X_i) and R(Y_i) be the ranks of X_i and Y_i. When there are no tied ranks, Spearman's coefficient simplifies to r_s = 1 - (6 ∑_{i=1}^n d_i^2) / [n (n^2 - 1)], where d_i = R(X_i) - R(Y_i). While Pearson's r measures the strength and direction of a strictly linear relationship, Spearman's r_s measures the strength and direction of any arbitrary monotonic relationship (whether linear or non-linear).
How does Spearman's correlation respond to non-linear monotonic transformations compared to Pearson's r?
Spearman's rank correlation is invariant under any strictly increasing monotonic transformation g: ℝ → ℝ (e.g., g(x) = exp(x), x^3, or ln(x)). If Y = g(X) where g is strictly increasing, the ranks of Y are identical to the ranks of X, yielding r_s = 1.000 exactly. In contrast, Pearson's r measures only straight-line linearity: if Y = exp(X), Pearson's r will drop below 1.0 (often to 0.7 - 0.8) because the exponential curve deviates from a straight line. Thus, Spearman's correlation provides a pure measure of monotonic co-movement free of functional form constraints.
What is the theoretical relationship between Pearson ρ and Spearman ρ_s for bivariate normal data?
For a bivariate normal distribution with linear correlation coefficient ρ, Karl Pearson (1907) proved that the theoretical expectation of Spearman's rank correlation is given by the exact formula: ρ_s = (6 / π) arcsin(ρ / 2). Expanding this in a Taylor series for small to moderate correlation: ρ_s ≈ (6 / π) [ρ/2 + (ρ/2)^3 / 6 + ...] = (3/π) ρ + (1/(8π)) ρ^3 ≈ 0.9549 ρ + 0.0398 ρ^3. For example, when ρ = 0.5, ρ_s = (6/π) arcsin(0.25) ≈ 0.4826. For small ρ, Pearson and Spearman correlations are closely concordant with a ratio of approximately 3/π ≈ 0.955.
How do leverage points and outliers affect Pearson r versus Spearman r_s?
Pearson's correlation has a breakdown point of 0%: introducing a single extreme leverage outlier (e.g., an observation placed at X = 100, Y = 100 in an otherwise uncorrelated cloud of points with r ≈ 0) can artificially inflate Pearson's r to near +1.0. Conversely, an outlier placed at X = 100, Y = -100 can flip a strong positive correlation to negative. Spearman's r_s is substantially more robust because raw values are replaced by integer ranks: an extreme outlier at X = 1,000,000 receives rank n, exactly 1 rank higher than the second largest value, severely bounding its leverage on the sum of squared rank differences.
How is the statistical significance of a correlation coefficient tested under the null hypothesis H_0: ρ = 0?
Under the null hypothesis H_0: ρ = 0 of zero correlation in a bivariate normal population, Ronald A. Fisher showed that the test statistic t = r √(n - 2) / √(1 - r^2) follows a Student's t-distribution with n - 2 degrees of freedom. For non-zero null hypotheses (H_0: ρ = ρ_0 ≠ 0), the distribution of r is highly skewed. Fisher developed the z-transformation: z = (1/2) ln((1 + r) / (1 - r)) = arctanh(r), which converges rapidly to a Gaussian distribution N(arctanh(ρ), 1/(n - 3)), enabling the construction of accurate confidence intervals.
What is Kendall's tau (τ) rank correlation, and how does it compare to Spearman's rho?
Maurice Kendall (1938) defined tau based on concordant and discordant pairs. A pair of observations (X_i, Y_i) and (X_j, Y_j) is concordant if (X_i - X_j)(Y_i - Y_j) > 0, and discordant if negative. Kendall's tau is τ = (C - D) / [n(n - 1)/2], where C is the number of concordant pairs and D is discordant pairs. While both τ and r_s measure monotonic association, Kendall's tau has a direct probabilistic interpretation: τ = P(concordance) - P(discordance). Furthermore, the sampling distribution of τ converges to normality much faster than that of Spearman's r_s in small samples.
Experimental precautions and statistical checks
- Ensure that paired observations (X_i, Y_i) represent genuine simultaneous measurements on the same experimental unit.
- Inspect scatter plots visually before interpreting Pearson's r to verify linearity and check for non-linear structures or clustering.
- Handle tied ranks appropriately by assigning mid-ranks (average ranks) to preserve rank sum properties.
- Perform replication across multiple independent pseudorandom seeds to verify that estimator variance and bias bounds are invariant to initial generator states.
Theoretical assumptions & limitations
- Pearson correlation measures only linear association: a deterministic quadratic relation Y = X² has r = 0 if centered symmetrically.
- Correlation does not imply causation: confounding latent variables can induce strong spurious correlations.