1. Education
You can opt-out at any time. Please refer to our privacy policy for contact information.

Logistic Regression

An Overview

By

Casual business people smiling in meeting
Caiaimage/Sam Edwards/OJO+/Getty Images
Logistic regression is a common statistical technique used in sociological studies. It provides a method for modeling a binary response variable, which takes values 0 and 1. Logistic regression is very similar to linear regression, however linear regression uses a continuous variable as the dependent variable in the model. In logistic regression, the dependent variable is binary. For example, if you were interested in looking at how race/ethnicity, education, and gender predicted respondents’ answers to a yes/no survey question, logistic regression would be a great method to use. Keep in mind that while the dependent variable is always binary, the independent, or predictor, variables can be either numerical or categorical.

How Is Logistic Regression Different From OLS Regression?

Besides the difference in the type of outcome variable being modeled (OLS predicts a continuous outcome variable whereas logistic regression predicts a dichotomous variable), there are several differences between OLS, or linear, regression and logistic regression.

First, one of the assumptions of OLS regression is that the error variances (residuals) are normally distributed. This is not the case with logistic regression. Second, with logistic regression, there is no standardized solution printed while with OLS, there is. And the unstandardized solution of logistic regression does not have the same straightforward interpretation as it does with OLS regression. One more difference is that in logistic regression, there is no r-square to gauge the variance accounted for in the overall model. Instead, a chi-square test is used to indicate how well the logistic regression model fits the data.

Goal of Logistic Regression

The goal of logistic regression is different than OLS because the dependent variable is not continuous. In logistic regression, we are predicting the likelihood that Y is equal to 1 (rather than 0) given certain values of X. That is, if X and Y have a positive linear relationship, the probability that a person will have a score of Y=1 will increase as values of X increase. So, instead of predicting the scores of the dependent variable as we do with OLS regression, we are instead predicting probabilities that an event will occur.

Interpreting the Coefficients

The coefficients in a logistic regression equation are more difficult to interpret. The saying in OLS that b represents “the change in Y with one unit change in X” is no longer applicable. Instead, we have to translate the coefficient using the exponent function. When we do that, we have a number that is pretty useful, called the odds ratio.

The odds ratio is equal to exp(B), or sometimes written eb. For example, if your results printout indicates the regression slope is 0.75, the odds ratio is 2.12 (because exp(.75)=2.12). This means that the probability that Y is equal to 1 is twice as likely (or 2.12 times) as the value of X is increased by one unit.

Using a specific example, let’s say we are looking at how gender predicted respondents’ answers to a yes/no survey question (with yes=1 and no=0). If the odds ratio for male is 2.12, this means that men are more than twice as likely to answer “yes” to the survey question compared to females.

Model Fit

Instead of using r-square as the statistic for overall model fit in logistic regression, we use a chi-square test to get the deviance. Chi-square is a test that measures the fit of the observed values to the expected values. The bigger the difference, or deviance, of the observed from the expected values, the poorer the fit of the model. Therefore we want a small deviance. As we add more variables to the equation, the deviance should get smaller, indicating an improvement of fit.

References

Newsom. (2010). Logistic Regression. http://www.upa.pdx.edu/IOA/newsom/da2/ho_logistic.pdf

©2014 About.com. All rights reserved.