# Model Selection Schema

There are various model selection criteria in use for picking variables in linear regression. Some are applicable to other models outside of linear regression.

Akaike Information Criterion

Akaike Information Criterion, or AIC, is a measure of strength of a given model at describing the data, relative to competing models. As such, if all models fit poorly, AIC won’t give any indication of that. AIC was developed by Hirutugu Akaike, in 1974. It draws its justification from information theory.

where is the number of coefficients being calculated in the model, and is the maximized value of the likelihood function of the model.

AIC effectively penalizes the model for having too many predictor variables. The “best fit” is the one that minimizes the AIC, because it offers the best tradeoff of maximizing the log-likelihood, and minimizing the number of predictors in the model. Additional predictors are only included if they offer enough information to justify their inclusion.

Coefficient of Determination

is the coefficient of determination. It is the percent of change in the dependent variable explained by change in the independent variables. We must first define some notation:

• is the mean of the observed values.
• is the predicted value for the dependent variable at the th observation.
• Sum of Squares – The Sums of squared values.
• is the total deviation from mean, or total variation in the data.
• is the total variation in the regression model. This is the value we are trying to maximize, with a theoretical ceiling at .
• is the total unexplained variation in the model. That is, the deviation from the predicted values. This is the value we are trying to minimize.

is an absolute measure of “Goodness of Fit” of a model. In order to get an absolute measure of fit that penalizes high numbers of predictors, we can use the Adjusted , denoted as .

Where , the degrees of freedom of the error term, and , the total degrees of freedom. (We subtract 1 degree of freedom for calculation of the mean, .)

is a useful criterion for comparing models and determining how well a given model represents the data. It penalizes adding more predictors to the model in a similar vein to AIC, but does so in a different manner. The models selected as “best” will not necessarily be the same, and AIC is generally preferred. In some disciplines (e.g., econometrics) both of these criteria are of limited use.