Regression and Correlation

Regression estimates how a dependent variable Y changes with independent variable(s) X. Simple linear regression models Y_i = a + b·X_i + e_i, where a is intercept (predicted Y at X=0), b is slope (change in predicted Y per unit change in X), and e_i is residual. Parameters are found by Ordinary Least Squares (OLS): minimize Σe_i² = Σ(Y_i - Ŷ_i)². OLS solution: b = Σ[(X_i - x̄)(Y_i - ȳ)] / Σ[(X_i - x̄)²] = Cov(X,Y)/Var(X), a = ȳ - b·x̄. The fitted line always passes through (x̄,ȳ). Correlation r measures strength and direction of linear association: r = Cov(X,Y)/(s_x·s_y) ∈ [-1,+1]. r does not equal slope, because slope scales with s_y/s_x: b = r·(s_y/s_x). r=0 means no linear association, but non-linear relation may still exist. Coefficient of determination R² = 1 - SS_res/SS_tot = r² in simple linear regression. It is proportion of variance in Y explained by X, not a measure of causality. Assumptions for valid inference: 1) linearity of conditional mean, 2) independent observations, 3) homoscedasticity (constant variance of residuals), 4) residuals approximately normal for small-n t-tests on b. Violations cause biased SEs or poor predictions. Correlation ≠ causation, sensitive to outliers, range restriction, and confounding. Multiple linear regression extends to Ŷ = a + b1·X1 + b2·X2 + ... with partial slopes holding other X constant. Engineers use regression for calibration curves, empirical material models, and trend estimation, but must check residual plots, not just R².