Backward Elimination is selection technique to remove those features that don’t have a significant effect on the dependent variable or prediction of output.
High p (>0.05) is not statistically significant.
Here are the stages in doing backward elimination:
- Select a significance level to stay in the model (ie: SL=0.05)
- Fit the full model with all possible predictors
- Consider the predictor with the highest P-value. If P > SL, go to step 4. Otherwhise, go to FIN
- Remove the predictor
- Fit the model without this variable -> Back again to step 3 to check if P > SL.
Program Link 