Describe correlation and regression with their uses and limitations.

Correlation

• The first step IS always the scatter diagram, plotted before any coefficient is calculated. It reveals the direction, the strength, whether the relationship is linear, and any outliers — none of which a single number can show
• The pearson correlation coefficient (r) ranges from -1 to +1
• R = +1 is a perfect positive linear relationship; r = -1 a perfect negative (inverse) one; and r = 0 indicates NO linear relationship
• Conventional interpretation — 0 to 0.25 little or none; 0.25 to 0.5 fair; 0.5 to 0.75 moderate to good; above 0.75 very good to excellent; but these are rules of thumb, and what counts as a strong correlation depends entirely on the field
• The sign indicates direction and the magnitude indicates strength; r = -0.8 is a stronger relationship than r = +0.5, which is a point students regularly get wrong
• The coefficient of determination (r²) is more informative than r itself — it gives the proportion of the variation in one variable explained by the other
• Its sobering implication — an r of 0.5, which sounds respectable, gives an r² of 0.25, meaning only 25% of the variation is explained and 75% is not
• The essential cautions about correlation, which are the substance of any good answer:
• 1. Correlation does not imply causation. An association may reflect chance, bias, confounding, reverse causation, or a genuine causal link
• The classic illustration — ice cream sales correlate with drowning deaths, both being caused by hot weather; neither causes the other
• 2. R measures only linear relationships. A strong curved relationship may give r close to zero — the U-shaped relationship between birth weight and perinatal mortality being a medical example, in which both low and high birth weight carry higher risk
• 3. R is highly sensitive to outliers, and a single extreme point can create or destroy an apparent correlation — which is precisely why the scatter plot must be inspected
• 4. A correlation found over a restricted range cannot be extrapolated beyond it
• 5. Statistical significance of r depends on sample size, so with a very large sample a trivial r of 0.05 may be "significant" and meaningless
• 6. The ecological fallacy — a correlation between group averages does not imply the same relationship in individuals
• Spearman’S rank correlation (rho) is the non-parametric alternative, used for ordinal data, non-normal data or where outliers are a concern; it works on the ranks rather than the values, and therefore detects any monotonic relationship, not only a linear one

Regression and its Interpretation

• Linear regression fits the equation Y = a + bX, where Y is the dependent (outcome) variable, X the independent (predictor) variable, a is the intercept (the value of Y when X is zero) and b is the regression coefficient or slope
• The slope b is what the analysis is usually for — it states the change in Y for a one-unit change in X, and is expressed in real, clinically interpretable units
• The line is fitted by the method of least squares, which minimises the sum of the squared vertical distances of the points from the line
• The differences between correlation and regression should be stated explicitly:
• Correlation measures the strength of association and is symmetrical — the correlation of X with Y equals that of Y with X, and neither variable is "dependent"
• Regression describes the form of the relationship and is not symmetrical — the regression of Y on X differs from that of X on Y, and one must decide which variable is the outcome
• Correlation is unit-free; regression coefficients carry units
• Only regression permits prediction
• The cautions about regression:
• Do not extrapolate beyond the range of the observed data; the relationship may not continue, and the intercept is frequently biologically meaningless (a birth weight at zero gestation, for instance)
• The relationship must be approximately linear, checked by plotting the residuals
• Regression to the mean is a distinct and important phenomenon — extreme values tend to be less extreme on RE-measurement, purely because part of the extremity was random
• Its clinical importance is considerable: a patient selected for a very high blood pressure will, on average, have a lower reading next time without any treatment. An uncontrolled before-and-after study will therefore show an apparent benefit that is entirely artefactual — which is one of the strongest arguments for a control group
• Multiple regression extends the model to several predictors, and its purpose in medical research is chiefly to adjust for confounding — giving the effect of each variable independent of the others
• Logistic regression is used where the outcome is binary (disease or no disease), which is the commonest situation in medicine; it yields adjusted odds ratios
• COX proportional hazards regression is used for time-to-event data, yielding hazard ratios
• The limitations of multivariable adjustment must be understood — it can only adjust for confounders that were measured and measured accurately; residual confounding remains from unmeasured or imperfectly measured factors; and adjusting for a variable that lies ON the causal pathway (a mediator) removes part of the very effect being studied
• To make the last point concrete — adjusting the relationship between obesity and heart disease for blood pressure will understate the effect of obesity, because raised blood pressure is one of the mechanisms by which obesity acts
• The overall lesson — no amount of statistical adjustment converts an observational study into a randomised one, and a "fully adjusted" model should be read with that firmly in mind
• Draw the scatter plot before calculating anything, since no single number shows shape, linearity or outliers
• A U-shaped relationship gives r near zero, as birth weight and perinatal mortality do, despite being strongly related
• One outlier can create or destroy a correlation, which is precisely why the plot must be inspected
• Regression to the mean flatters every uncontrolled study, since extreme values become less extreme on re-measurement
• A patient picked for high blood pressure improves at the next visit without any treatment at all
• R squared is more informative than r; an r of 0.5 explains only a quarter of the variation
• Minus 0.8 is stronger than plus 0.5, since sign gives direction and magnitude gives strength
• Ice cream and drowning share a confounder, which is the standard illustration that correlation is not causation
• Correlation is symmetrical and regression is not, which is the cleanest distinction between the two
• Only regression permits prediction, and its coefficients carry units while r is unit-free
• The slope states the change in Y per unit of X, which is the clinically interpretable quantity
• Do not extrapolate beyond the observed range, where the relationship may not hold and the intercept may be meaningless
• Spearman detects any monotonic relationship, not only a linear one, and suits ordinal or skewed data
• Least squares minimises squared vertical distances, which is how the line is fitted
• Multiple regression adjusts for confounding, giving each effect independent of the others measured
• Logistic regression suits binary outcomes and yields adjusted odds ratios, which is the common medical case
• Cox regression handles time to event and yields hazard ratios, accommodating censored observations
• Adjustment reaches only measured confounders, and residual confounding always remains
• Never adjust for a mediator; doing so removes part of the very effect being studied
• Adjusting obesity for blood pressure understates it, since raised pressure is one mechanism by which obesity acts
• No adjustment makes an observational study randomised, which is how a fully adjusted model should be read
• Significance of r depends on sample size, so a trivial correlation may be significant in a very large study
• Plot the residuals to check linearity, which reveals curvature that the raw scatter may obscure
• A correlation over a restricted range cannot be extrapolated beyond the values actually observed