Cheatsheet: Regression & Correlation

1. Simple Linear Regression

1.1 Model Equation

• β₀ = population intercept, β₁ = population slope
• b₀ = sample intercept, b₁ = sample slope
• ε = random error term with E(ε) = 0, Var(ε) = σ²

1.2 Least Squares Estimation

1.3 Sum of Squares Decomposition

1.4 Mean Square Error and Standard Error

• Degrees of freedom for error: df = n - 2

2. Correlation Analysis

2.1 Correlation Coefficient

2.2 Interpretation of Correlation

• r = +1: Perfect positive linear relationship
• r = -1: Perfect negative linear relationship
• r = 0: No linear relationship
• |r| > 0.8: Strong correlation
• 0.5 < |r|="">< 0.8:="" moderate="">
• |r| < 0.5:="" weak="">
• Correlation measures only linear relationships

2.3 Coefficient of Determination

• 0 ≤ R² ≤ 1
• R² = 0.75 means 75% of variation in Y is explained by the regression model

2.4 Hypothesis Test for Correlation

3. Inference for Regression Parameters

3.1 Confidence Intervals

3.2 Hypothesis Test for Slope

3.3 ANOVA for Regression

• Test H₀: β₁ = 0 using F = MSR/MSE
• Reject H₀ if F > F(α, 1, n-2)
• F-test and t-test for slope are equivalent: F = t²

4. Prediction and Confidence Intervals

4.1 Point Prediction

4.2 Confidence Interval for Mean Response

4.3 Prediction Interval for Individual Response

• Prediction interval is wider than confidence interval
• Both intervals are narrowest at x₀ = x̄

5. Regression Assumptions

5.1 Model Assumptions (LINE)

5.2 Residual Analysis

5.3 Common Violations and Patterns

• Funnel shape: Non-constant variance (heteroscedasticity)
• Curved pattern: Non-linearity; may need transformation
• Outliers: Points far from pattern; high residual values
• Influential points: Large effect on regression line; check leverage and Cook's distance

6. Multiple Linear Regression

6.1 Model Equation

• k = number of predictor variables
• Parameters estimated using least squares (matrix methods)

6.2 Multiple Regression ANOVA

6.3 Adjusted R²

• R²adj penalizes for adding predictors that do not improve model
• Use R²adj for comparing models with different numbers of predictors

6.4 Individual Predictor Tests

6.5 Multicollinearity

• High correlation among predictor variables
• Inflates standard errors of coefficients
• Detection: Variance Inflation Factor (VIF) > 10 indicates problem
• VIF = 1/(1 - R²ⱼ) where R²ⱼ is R² from regressing Xⱼ on other predictors

7. Model Selection and Diagnostics

7.1 Variable Selection Methods

7.2 Model Selection Criteria

7.3 Leverage and Influence

8. Transformations

8.1 Common Transformations

8.2 Special Regression Models

9. Key Formulas Summary

9.1 Quick Reference