The more features a model has, the more sensitive the model is to errors due to variance. Thus, we want to select the minimum required features to produce a valid model.
A common approach to eliminate features is to eliminate the ones that are the least important to the model. Then we re-evaluate if the model actually performs better during cross-validation.
Yellowbrick’s FeatureImportances
is ideal for this task since it helps us to visualize the relative importance of the features for the model.