Regression

Regression is the process of fitting a mathematical model to a set of measured data so that the model can describe the trend of the data, predict new values, or estimate physical parameters. In engineering, measurements are always affected by noise, so an exact fit through every data point is neither possible nor desirable. Instead, regression minimizes a chosen error measure between the observed values y_i and the model predictions y_hat_i. The most common approach is least-squares regression, which minimizes the sum of squared residuals SSE = Σ(y_i - y_hat_i)². For a straight-line model y = a + bx, the least-squares solution has a closed form: b = (nΣx_i y_i - Σx_i Σy_i) / (nΣx_i² - (Σx_i)²) and a = (Σy_i - bΣx_i)/n. For a nonlinear model, the fit is usually obtained iteratively by Gauss-Newton or Levenberg-Marquardt methods, which linearize the model about the current parameter estimates. Goodness of fit is judged by the coefficient of determination R² = 1 - SSE/SST, where SST = Σ(y_i - y_mean)², by the standard error of the estimate, and by residual plots. R² alone can be misleading: adding parameters always increases R², so an adjusted R² or a validation data set is preferred. Overfitting occurs when the model has too many parameters relative to the number of data points and begins to fit noise rather than the underlying trend. Underfitting occurs when the model is too simple to capture the physical behavior. Multicollinearity, outliers, and heteroscedasticity (non-constant variance) also affect the reliability of the fitted parameters. In engineering, regression is used to calibrate sensors, fit material constitutive models, derive empirical correlations from experimental data, and build surrogate models for expensive simulations. The fitted parameters should always be checked against physical units and expected ranges; a mathematically good fit that predicts an unrealistic modulus or negative stiffness is not physically meaningful.