Building Models
Sociology 312/412/512, University of Oregon
Drawing straight lines
Drawing straight lines
Elements of a straight line
The OLS regression line as a model
The OLS regression line is often called a linear model because we are measuring the relationship between two variables by applying a linear function to characterize the relationship.
the lm command can be used to create a model object in R:
model <- lm(property_rate~unemploy_rate, data=crimes)The tilde (~) is used to indicate the relationship between the two variables with the dependent variable on the left hand side.
I can then use the coef command on this model to get my coefficients (i.e. intercept and slope).
coef(model) (Intercept) unemploy_rate
1238.0725 220.7796 Add the best-fitting line to your scatterplot
ggplot(crimes, aes(x=unemploy_rate, y=property_rate))+
geom_point()+
geom_smooth(method="lm", se=FALSE)+
labs(x="unemployment rate",
y="property crimes (per 100,000)")geom_smooth with the argument method="lm" will add the OLS regression line to your scatterplot.
se=FALSE will suppress a confidence band which I will show later.
Nonsensical Intercepts
Try interpreting these numbers from a regression model where the dependent variable is sexual frequency (sexual encounters per year) and the independent variable is age in years.
\[\hat{\texttt{sex}}_i=88.32-0.83(\texttt{age}_i)\]
Intercept
The model predicts that newborns will have sex 88.32 times per year, on average.
😮 Say what??!!
Getting meaningful intercepts
Lets subtract some constant \(a\) from the variable \(x\):
\[x^*=x-a\]
The value for zero on our new re-centered \(x^*\) will be \(a\) on the original scale.
In the formula of the lm command in R, we can do this easily by surrounding our math with I() which tells R to apply the function inside and treat it as a new variable:
model <- lm(sexf~I(age-18), data=sex)
round(coef(model), 2)(Intercept) I(age - 18)
73.45 -0.83 
How good is \(x\) as a predictor of \(y\)?
I pick a random observation from the dataset and ask you to guess the value of \(y\). What is your best guess?
Choose \(\bar{y}\)
- Because it is the balancing point, the mean will give you the smallest error, on average.
- If you repeat this procedure, your average error in prediction will be equal to \(s_y\).
How good is \(x\) as a predictor of \(y\)?
I pick a random observation from the dataset and tell you its value of \(x\), and then ask you to guess the value of \(y\). What is your best guess?
Choose \(\hat{y}_i\) from the linear model
- Assuming that a linear model is reasonable, the predicted value from this model will be your best guess.
- The average error in your prediction will be equal to the average residual from the model, \(|\hat{y}_i-y_i|\).
How much did we reduce the error?
On average, what proportion of the red line is the green line across all observations?
Red: \(\sum_{i=1}^n (y_i-\bar{y})^2\)
Green: \(\sum_{i=1}^n (y_i-\hat{y}_i)^2\)
Proportion: \(1-\frac{\sum_{i=1}^n (y_i-\hat{y}_i)^2}{\sum_{i=1}^n (y_i-\bar{y})^2}\)
It turns out:
\[1-\frac{\sum_{i=1}^n (y_i-\hat{y}_i)^2}{\sum_{i=1}^n (y_i-\bar{y})^2}=r^2\]
cor(crimes$unemploy_rate, crimes$property_rate)^2[1] 0.1608377⚠️ Linear models only fit straight lines
⚠️ Outliers can be influential
⚠️ Don’t extrapolate beyond range of data
Poverty might be a confounding variable
Causal
Spurious
What is the effect of education on crime?
The relationship seemed positive…
but is was negative once we controlled for the poverty rate!
Make a scatterplot with indicator
Make a scatterplot with indicator
Make a scatterplot with indicator
Adding context to a relationship
ggplot(earnings, aes(x=nchild, y=wages,
color=gender))+
geom_jitter(alpha=0.05)+
scale_color_viridis_d(end=0.75)+
geom_smooth(method="lm", se=FALSE)+
labs(x="number of children",
y="hourly wages in USD")
Additive models make parallel lines
\[\begin{eqnarray*} \hat{\texttt{wages}}_i&=&25.24+1.55(\texttt{nchild}_i)\\ & & -3.97(\texttt{female}_i) \end{eqnarray*}\]
- The effect of number of children on wages ($1.55) is forced to be the same for men and women.
- The wage difference between men and women ($3.97) is forced to be the same at all values of number of children.
Multiplicative models give non-parallel lines
\[\begin{eqnarray*}\hat{\texttt{wages}}_i & = & 24.72+1.78(\texttt{nchild}_i)-2.84(\texttt{female}_i)\\ & & -1.33(\texttt{nchild}_i)(\texttt{female}_i)\\\end{eqnarray*}\]
- This model shows that men and women get different returns to wages for the number of children with men getting a much greater return ($1.78 to $0.45).
- This models shows that the wage gap starts at $2.84 when men and women have no children and grows by $1.33 for every child.
Same underlying reality
