table(politics$party)
Democrat Republican Independent Other
1472 1236 1381 148 Looking at Distributions
Sociology 312/412/512, University of Oregon
Aaron Gullickson
What is a Distribution?
The concept of a distribution
When we refer to the distribution of a variable, we are referring to how the different values of that variable are distributed across the given observations.
Look at it
- We can make a plot that shows the distribution.
- We make different kinds of plots for categorical and quantitative variables.
- Barplots for categorical variables
- Histograms for quantitative variables
Measure it
- We can calculate summary measures of the center and spread of the distribution.
- We an only calculate summary measures for quantitative variables.
Calculating frequencies
In order to display the distribution of a categorical variable, we first need to calculate the frequency which is the number of observations that belong to each possible category. We can do this easily in R with the table command:
Lets convert these frequencies into proportions by dividing through by the total number of observations. We can also do this easily in R by adding the sum command to the previous command:
Proportions and percents
R also has a built-in function called prop.table that will calculate proportions automatically. We just need to feed the output of the table command into it.
Democrat Republican Independent Other
0.34741562 0.29171584 0.32593816 0.03493038 We can employ this “wrapping” feature of R to do some more tidying up. In this case, I want to round the number of digits and multiply by 100 to turn my proportions into percents. I also use the sort command to sort values from highest to lowest.
How can we plot the percent?
Don’t use a piechart
Use a barplot
Constructing a barplot using ggplot
We will use ggplot to construct graphs.
In a ggplot, multiple commands are linked together with + signs.
The first command of ggplot takes two arguments. The first argument is the data we want to use (in this case, the politics dataset). The second argument is the aes command that defines aesthetics for the full plot.
The second command is geom_bar.All plots require some kind of “geometry” command which in this case makes bars.
scale_y_continuous(label=scales::percent) causes my proportions on the y-axis to be reported as percents.
The labs command can be used to add nice labeling of axes and to create titles and captions.
theme_bw defines a theme for the overall plot. I prefer theme_bw to the default theme in ggplot.
Code and output
Visualize quantitative variables with a histogram
How a histogram is created
- We break the variable into equivalent intervals called bins. For a histogram of movie runtime length, we might use bins of 5 minutes width, so our bins would look like 90-94 minutes, 95-99 minutes, 100-104 minutes, 105-109 minutes, etc.
- We calculate the frequency of observations that fall into each bin. Technically, we need to decide which bin to put cases that straddle two bins (e.g. exactly 5 minutes). R defaults to putting these cases in the lower category.
- We make a barplot of these frequencies, but we put no space between the bars.
Code and output for making a histogram
What are we looking for in a histogram?
- Shape
- Is it symmetric or skewed?
- Center
- Where is the center or peak of the distribution and is there only one?
- Spread
- How spread out are the values around the center?
- Outliers
- Are there any observations that have relatively very high or low values?
The Center of a Distribution
What does “center” mean?
Mean
The mean is the balancing point of a distribution. Imagine trying to put a column underneath a histogram so that it does not tip one direction or the other. This balancing point is the mean.
Median
The median is the midpoint of the distribution. At this point, 50% of the observations have lower values, and 50% have higher values.
Mode
The mode is the high point of the distribution, or the peak. It is typically much less useful than the other two measures.
Calculating the mean
The mean (represented mathematically as
😱 Equations??!!
Don’t panic! We will walk through what these symbols mean.
- We use the
Calculate the mean in R
To calculate the mean we just sum up all the values of sum command will sum up a variable and the nrow command will give us the number of observations, so:
The mean move runtime is 106.8 minutes.
Calculating the median
We just need to sort the observations from smallest to largest and pick the exact middle value of the distribution.
- If there are an odd number of observations, there will always be an exact midpoint.
- If we have an even number of observations, we have to take the two values closest to the midpoint and take their mean.
With an odd number of 4343 movies, the exact midpoint is the 2172nd movie. We can use the sort command to sort and then extract the 2172nd movie by using square brackets:
Why are the mean and median different?
- In perfectly symmetric distributions, the mean and the median will be the same. In other words, the balancing point will be at the midpoint.
- Skewness will “pull” the mean in the direction of the skew, but not the median. This is because the mean will need to move in that direction to maintain balance.
Skewness can create large differences
Modal Categories
- We can’t generally calculate measures of center for categorical variables.
- Since the mode is the most common observation, we can make an exception for this case.
- The modal category is the most frequent category in a categorical variable.
Percentiles and the Five-Number Summary
Percentiles/Quantiles
- A given percentile tells you what percent of the distribution is below that number.
- We have already seen one example of a percentile: the median. The median is the 50th percentile. 50% of the observations are below this value.
- Percentiles are sometimes also called quantiles, but I will use the term percentile in this course.
Calculate percentiles in R
The quantile command in R will calculate a given percentile.
To calculate the percentile with this command, we need to add a second argument called probs where we feed in a list of proportions. So if we wanted to calculate the 13th and 76th percentile of movie runtime:
13% of movies are 90 minutes or shorter and 76% of movies are 116 minutes or shorter.
The five-number summary
If I run the quantile command without the probs argument, I get:
- The 25th percentile, 50th percentile, and 75th percentile are called the quartiles because they split the data into four equal quarters.
- When combined with the minimum (0%) and maximum (100%), they create the five-number summary.
Anatomy of the boxplot
Code and output for boxplot
The x="" in the aesthetics is not necessary but does create a nicer looking x axis. Your variable is assigned to y.
You can use fill to determine color of the box and outlier.color to determine color of individual points.
Measuring the Spread of a Distribution
Distributions can vary in their spread
Measures of spread
Interquartile Range
The Interquartile Range (IQR) is the distance between the 25th and 75th percentile. It can be calculated by the IQR command in R:
The 75th percentile of movie runtime is 21 minutes longer than the 25th percentile.
Variance and Standard Deviation
Standard deviation (SD) is the most common measure of spread in a variable. Loosely, standard deviation measures the average distance from the mean of all observations. Variance is the squared value of standard deviation.
Calculating the standard deviation
The standard deviation
😮 More Equations??!!
Don’t panic! We will go through it one step at a time.
Calculating the standard deviation
The average movie is about 16.3 minutes away from the mean movie runtime.
Sociology 312/412/512, University of Oregon
Looking at Distributions Sociology 312/412/512, University of Oregon Aaron Gullickson