Can I use geom_label_repel() with maps?

You learned about the {ggrepel} package in session 9, with its geom_text_repel() and geom_label_repel() functions that make sure none of your labels overlap:

library(tidyverse)
library(sf)
library(ggrepel)

small_mpg <- mpg %>% 
  # Only use the first 10 rows
  slice(1:10) %>% 
  # Make a label column
  mutate(fancy_label = paste0(manufacturer, " ", model, " (", year, ")"))

ggplot(small_mpg, aes(x = displ, y = hwy, color = drv)) +
  geom_point() +
  geom_label_repel(aes(label = fancy_label), seed = 1234)

In session 12, you learned about geom_sf_text() and geom_sf_label() for adding text and labels to maps. But what if your map labels overlap, like this?

# Download cb_2022_us_county_5m.zip under "County" from
# https://www.census.gov/geographies/mapping-files/time-series/geo/cartographic-boundary.html
ga_counties <- read_sf("data/cb_2022_us_county_5m/cb_2022_us_county_5m.shp") %>% 
  filter(STATEFP == 13)

ga_places <- tribble(
  ~city, ~lat, ~long,
  "Atlanta", 33.748955, -84.388099,
  "Alpharetta", 34.075318, -84.294105,
  "Duluth", 34.002262, -84.143614
) %>% 
  st_as_sf(coords = c("long", "lat"), crs = st_crs("EPSG:4326"))

ggplot() +
  geom_sf(data = ga_counties, linewidth = 0.1) +
  geom_sf(data = ga_places) +
  geom_sf_label(data = ga_places, aes(label = city)) +
  theme_void()

Unfortunately there’s no such thing as geom_sf_label_repel(). BUT there’s still a way to use geom_label_repel() and geom_text_repel() with maps, with a couple little tweaks:

1. You have to map the geometry column in the data to the geometry aesthetic in geom_text/label_repel()
2. You have to tell geom_text/label_repel() to use the “sf_coordinates” stat so that it uses the latitude and longitude coordinates for x/y

ggplot() +
  geom_sf(data = ga_counties, linewidth = 0.1) +
  geom_sf(data = ga_places) +
  geom_label_repel(
    data = ga_places,
    aes(label = city, geometry = geometry),
    stat = "sf_coordinates", seed = 1234
  ) +
  theme_void()

I tried to make a map and countries are missing—why?

Many of you were brave and made a map of refugee counts for mini project 2. That’s fantastic!

If you did, you likely ran into an issue with plotting the countries and getting an incomplete map. Here’s an example with our beloved gapminder data.

library(countrycode)  # For dealing with country names, abbreviations, and codes
library(gapminder)    # Global health and wealth

# Add an ISO country code column to gapminder for joining
gapminder_clean <- gapminder %>% 
  mutate(ISO_A3 = countrycode(country, "country.name", "iso3c"))

# Load the world map data from exercise 12
# Download "Admin 0 – Countries" from
# https://www.naturalearthdata.com/downloads/110m-cultural-vectors/
world_map <- read_sf("data/ne_110m_admin_0_countries/ne_110m_admin_0_countries.shp") %>% 
  filter(ISO_A3 != "ATA") %>%  # Bye penguins
  mutate(ISO3 = ADM0_A3)  # Use ADM0_A3 as the main country code column

Let’s take just 2007 from gapminder and map life expectancy. To do this we’ll need to combine or join the two datasets. One logical way to do this would be to take gapminder, join the world map data to it, and then plot it:

gapminder_with_map <- gapminder_clean %>% 
  filter(year == 2007) %>% 
  left_join(world_map, by = join_by(ISO3))

ggplot() +
  geom_sf(data = gapminder_with_map, aes(fill = lifeExp))
## Error in `geom_sf()`:
## ! Problem while computing stat.
## ℹ Error occurred in the 1st layer.
## Caused by error in `compute_layer()`:
## ! `stat_sf()` requires the following missing aesthetics: geometry

oh no there’s an error! When we joined the map data, the special attributes of the geometry column in world_map got lost. The column is still there, but it won’t automatically plot with geom_sf(). We can fix that by specifying that the column named “geometry” does indeed contain all the geographic data with st_set_geometry():

gapminder_with_map <- gapminder_clean %>% 
  filter(year == 2007) %>% 
  left_join(world_map, by = join_by(ISO3)) %>% 
  # Fix the geometry column
  st_set_geometry("geometry")

ggplot() +
  geom_sf(data = gapminder_with_map, aes(fill = lifeExp)) +
  theme_void()

We have a… map? It’s missing a bunch of countries (Russia is the most glaringly obvious hole!). That’s because those countries aren’t in gapminder, so their corresponding maps didn’t come over when using left_join(). We can confirm by counting rows. The original map data has maps for 176 countries. Gapminder has 142 countries in 2007. The combined gapminder_with_map dataset only has 142 rows—we’re not plotting 34 countries, since they’re not in gapminder.

nrow(world_map)
## [1] 176
nrow(gapminder_clean %>% filter(year == 2007))
## [1] 142
nrow(gapminder_with_map)
## [1] 142

One quick and easy way to fix this is to use two geom_sf() layers: one with the whole world and one with the partial gapminder-only map:

ggplot() +
  geom_sf(data = world_map) +
  geom_sf(data = gapminder_with_map, aes(fill = lifeExp)) +
  theme_void()

The better way to fix this is to join the two datasets in a different order—start with the full map data and then add gapminder to it. This maintains the specialness of the geometry column and keeps all the original rows in world_map. For countries that are in the map data but not in gapminder, they’ll still be in the final map_with_gapminder data, but they’ll have NA for life expectancy:

map_with_gapminder <- world_map %>% 
  left_join(filter(gapminder_clean, year == 2007), by = join_by(ISO3))

ggplot() +
  geom_sf(data = map_with_gapminder, aes(fill = lifeExp)) +
  theme_void() +
  # Make the countries with missing data a different color
  scale_fill_gradient(na.value = "grey90")

What if we want to facet though? This is just one year—what if we want to show panels for multiple years? This gets a little tricky. The gapminder data has rows for different country/year combinations (Afghanistan 1952, Afghanistan 1957, Albania 1952, etc.), but the world map data only has rows for countries. If we join the gapminder data to the world map data and gapminder has multiple rows for years, there’s no clear place for the gapminder rows to connect with the world map rows. R will try to make it work and repeat world_map rows for each of the repeated years, but it can be unpredictable.

The best approach I’ve found for doing this is to create what I call a “skeleton” data frame that has all the possible combinations of (1) unique countries in the map data and (2) unique years in gapminder (or the refugee data if you’re using that). The expand_grid() function does this automatically. Like, look what happens if we tell it to make rows for every combination of A, B, C and 1, 2, 3—we get A1, A2, A3, B1, B2, and so on:

expand_grid(column_1 = c("A", "B", "C"),
            column_2 = c(1, 2, 3))
## # A tibble: 9 × 2
##   column_1 column_2
##   <chr>       <dbl>
## 1 A               1
## 2 A               2
## 3 A               3
## 4 B               1
## 5 B               2
## 6 B               3
## 7 C               1
## 8 C               2
## 9 C               3

We’ll make a similar skeleton with all the countries in the map and all the years we care about in gapminder. We’ll just show two panels—1952 and 2007—so we’ll make a little filtered dataset first. Then we’ll use expand_grid() to make a dataset with all those combinations: Afghanistan 1952, Afghanistan 2007, Albania 1952, Albania 2007, and so on:

gapminder_smaller <- gapminder_clean %>%
  filter(year %in% c(1952, 2007))

skeleton <- expand_grid(ISO3 = unique(world_map$ISO3),
                        year = unique(gapminder_smaller$year))
skeleton
## # A tibble: 352 × 2
##    ISO3   year
##    <chr> <int>
##  1 FJI    1952
##  2 FJI    2007
##  3 TZA    1952
##  4 TZA    2007
##  5 SAH    1952
##  6 SAH    2007
##  7 CAN    1952
##  8 CAN    2007
##  9 USA    1952
## 10 USA    2007
## # ℹ 342 more rows

Neat, that works. There’s Fiji in 1952 and 2007, Tanzania in 1952 and 2007, and so on. Those are all the possible countries in world_map with all the possible years in gapminder_smaller.

Next we can join in the gapminder data for each country and year, and join in the map data for each country. Notice how it has the same number of rows as skeleton (352). If a country doesn’t have gapminder data (like Fiji here), it gets an NA for lifeExp and pop and gdpPercap. But it still has map data for both 1952 and 2007, so it’ll show up in a plot.

full_gapminder_map <- skeleton %>% 
  left_join(gapminder_smaller, by = join_by(ISO3, year)) %>%
  left_join(world_map, by = join_by(ISO3)) %>% 
  # The geometry column lost its magic powers after joining, so add it back
  st_set_geometry("geometry")
full_gapminder_map
## Simple feature collection with 352 features and 175 fields
## Geometry type: MULTIPOLYGON
## Dimension:     XY
## Bounding box:  xmin: -180 ymin: -55.61183 xmax: 180 ymax: 83.64513
## Geodetic CRS:  WGS 84
## # A tibble: 352 × 176
##    ISO3   year country  continent lifeExp     pop gdpPercap featurecla scalerank
##    <chr> <int> <fct>    <fct>       <dbl>   <int>     <dbl> <chr>          <int>
##  1 FJI    1952 <NA>     <NA>         NA   NA            NA  Admin-0 c…         1
##  2 FJI    2007 <NA>     <NA>         NA   NA            NA  Admin-0 c…         1
##  3 TZA    1952 Tanzania Africa       41.2  8.32e6      717. Admin-0 c…         1
##  4 TZA    2007 Tanzania Africa       52.5  3.81e7     1107. Admin-0 c…         1
##  5 SAH    1952 <NA>     <NA>         NA   NA            NA  Admin-0 c…         1
##  6 SAH    2007 <NA>     <NA>         NA   NA            NA  Admin-0 c…         1
##  7 CAN    1952 Canada   Americas     68.8  1.48e7    11367. Admin-0 c…         1
##  8 CAN    2007 Canada   Americas     80.7  3.34e7    36319. Admin-0 c…         1
##  9 USA    1952 United … Americas     68.4  1.58e8    13990. Admin-0 c…         1
## 10 USA    2007 United … Americas     78.2  3.01e8    42952. Admin-0 c…         1
## # ℹ 342 more rows
## # ℹ 167 more variables: LABELRANK <int>, SOVEREIGNT <chr>, SOV_A3 <chr>,
## #   ADM0_DIF <int>, LEVEL <int>, TYPE <chr>, TLC <chr>, ADMIN <chr>,
## #   ADM0_A3 <chr>, GEOU_DIF <int>, GEOUNIT <chr>, GU_A3 <chr>, SU_DIF <int>,
## #   SUBUNIT <chr>, SU_A3 <chr>, BRK_DIFF <int>, NAME <chr>, NAME_LONG <chr>,
## #   BRK_A3 <chr>, BRK_NAME <chr>, BRK_GROUP <chr>, ABBREV <chr>, POSTAL <chr>,
## #   FORMAL_EN <chr>, FORMAL_FR <chr>, NAME_CIAWF <chr>, NOTE_ADM0 <chr>, …

Now we can plot it and we’ll have consistent countries in each panel:

ggplot() +
  geom_sf(data = full_gapminder_map, aes(fill = lifeExp)) +
  facet_wrap(vars(year), ncol = 1) +
  scale_fill_gradient(na.value = "grey90") +
  theme_void()

Perfect!

Saving data that takes a long time to make

In these later sessions, I’ve had you do things with data from different places on the internet. In session 13 you’ll grab books from Project Gutenberg. Some of you used osrmRoute() in exercise 12 to create a mapped route between cities. Some of you used {tidygeocoder} to geocode addresses in exercise 12. In past sessions you’ve used WDI() to download data from the World Bank.

When you knit a document, R starts with a brand new empty session without any packages or data loaded, and then it runs all your chunks to load packages, load data, and run all your other code. If you have code that grabs data from the internet, it will run every time you knit your document. Remember my suggestion to knit often? You’ll re-download the data, re-create routes, re-geocode addresses, and so on every time you keep re-knitting. This is excessive, slow, and—most especially—bad R etiquette. You don’t want to keep accessing those servers and recalculate things and redownload things you don’t need to update.

BUT at the same time, you should care about reproducibility. You want others—and future you—to be able to run your code and create the same plots and tables and get the same data. But you don’t want to do all that excessively and impolitely.

The solution is to be a little tricky with your R Markdown file. If you have code that needs to grab something from the internet, put it in a chunk that doesn’t run—use eval=FALSE in its chunk options. Then, in an invisible chunk (with include=FALSE) load the pre-downloaded data manually. I showed this in example 8 (with {WDI}) and example 11 (with {tidyquant}) and example 13 (with {gutenberger})

Here’s a quick basic example with Project Gutenberg book data. There are two chunks: get-book-fake and load-book-real:

```{r get-book-fake, eval=FALSE}
little_women_raw <- gutenberg_download(514, meta_fields = "title")
```

```{r load-book-data-real, include=FALSE}
little_women_file <- "data/little_women_raw.csv"

if (file.exists(little_women_file)) {
  little_women_raw <- read_csv(little_women_file)
} else {
  little_women_raw <- gutenberg_download(514, meta_fields = "title")
  
  write_csv(little_women_raw, little_women_file)
}
```

1. The first chunk (get-book-fake) contains the code for downloading data with gutenberg_download(). It will appear in the document, but it will not run since it has eval=FALSE on. It will not try to grab anything from the internet. If someone were to follow along with the code in your document, they could run that code and get the book (good!), and it won’t run repeatedly on your end (also good!).
2. The second chunk (load-book-data-real) does a neat little trick. It first checks to see if a CSV file named data/little_women_raw.csv exists. If it does, it’ll just load it with read_csv(). If it doesn’t, it’ll grab data from the internet with gutenberg_download() and then it will save that as data/little_women_raw.csv. This is really neat stuff. If you’re knitting your document for the first time, you won’t have the Little Women data yet, so the code will connect to Project Gutenberg and get it. The code will then save that data to your computer as a CSV file. The next time you knit, R won’t need to connect to the internet again—it’ll load the CSV file instead of grabbing it from Project Gutenberg. You can knit as many times as you want—you won’t need to reconnect to any remote servers again.

Again, the general pattern for this is to create two chunks: (1) a fake one that people will see in the document but won’t run, and (2) a real one that will run and load data locally if it exists, but that people won’t see.

```{r get-book-fake, eval=FALSE}
little_women_raw <- gutenberg_download(514, meta_fields = "title")
```

```{r load-book-data-real, include=FALSE}
little_women_file <- "data/little_women_raw.rds"

if (file.exists(little_women_file)) {
  little_women_raw <- readRDS(little_women_file)
} else {
  little_women_raw <- gutenberg_download(514, meta_fields = "title")
  
  saveRDS(little_women_raw, little_women_file)
}
```

Warnings and messages

Your knitted document has warnings and package loading messages.

You should turn off those warnings and messages. See this and this for more about how.

ALL CAPS

Consider converting the borough names to NOT ALL CAPS, since you don’t repeat all caps anywhere else

In the data, the borough names are all spelled with ALL CAPS:

rats_clean %>% 
  select(created_date, borough)
## # A tibble: 101,913 × 2
##    created_date        borough      
##    <dttm>              <chr>        
##  1 2015-09-04 00:00:00 MANHATTAN    
##  2 2015-09-04 00:00:00 STATEN ISLAND
##  3 2015-09-04 00:00:00 STATEN ISLAND
##  4 2015-09-04 00:00:00 BROOKLYN     
##  5 2015-09-04 00:00:00 BRONX        
##  6 2015-09-04 00:00:00 BROOKLYN     
##  7 2015-09-04 00:00:00 QUEENS       
##  8 2015-09-04 00:00:00 BROOKLYN     
##  9 2015-09-04 00:00:00 MANHATTAN    
## 10 2015-09-04 00:00:00 STATEN ISLAND
## # ℹ 101,903 more rows

You can convert that to not all caps with one of two functions from the {stringr} package (which is one of the nine that R loads when you run library(tidyverse)): str_to_title() changes text to title case (where each word is capitalized) while str_to_sentence() changes text to sentence case (where the first letter in a sentence is capitalized):

str_to_title("OH NO! RATS IN NEW YORK")
## [1] "Oh No! Rats In New York"
str_to_sentence("OH NO! RATS IN NEW YORK")
## [1] "Oh no! Rats in new york"

You can use it with mutate():

rats_clean <- rats_clean %>% 
  mutate(borough = str_to_title(borough))

rats_clean %>% 
  select(created_date, borough)
## # A tibble: 101,913 × 2
##    created_date        borough      
##    <dttm>              <chr>        
##  1 2015-09-04 00:00:00 Manhattan    
##  2 2015-09-04 00:00:00 Staten Island
##  3 2015-09-04 00:00:00 Staten Island
##  4 2015-09-04 00:00:00 Brooklyn     
##  5 2015-09-04 00:00:00 Bronx        
##  6 2015-09-04 00:00:00 Brooklyn     
##  7 2015-09-04 00:00:00 Queens       
##  8 2015-09-04 00:00:00 Brooklyn     
##  9 2015-09-04 00:00:00 Manhattan    
## 10 2015-09-04 00:00:00 Staten Island
## # ℹ 101,903 more rows

(I’d use str_to_title() here since Staten Island is two words; if you use str_to_sentence() it’ll turn into “Staten island”.)

Now your plots will have nicer borough names:

rats_summarized <- rats_clean %>% 
  group_by(borough) %>% 
  summarize(total = n())

ggplot(rats_summarized, aes(x = borough, y = total)) +
  geom_col()

Sorting and ordering

Consider sorting the locations by number of sightings instead of alphabetically

In the plot above, the boroughs on the x-axis are in alphabetic order. If we want to tell a better story, though, it’s helpful to reorder them so that we can more easily see which boroughs have the most and least rats. See here for more about reordering categories. We can sort the data and then use fct_inorder() from the {forcats} package (also one of the nine that gets loaded with library(tidyverse)) to lock these borough names in the right order:

rats_summarized <- rats_clean %>% 
  group_by(borough) %>% 
  summarize(total = n()) %>% 
  # Sort by total in descending order
  arrange(desc(total)) %>% 
  # Lock bhe borough names in place
  mutate(borough = fct_inorder(borough))

ggplot(rats_summarized, aes(x = borough, y = total)) +
  geom_col()

Unbalanced facets

The facets are a little unbalanced given that there are 8 panels. Consider adding… something?… to that empty panel, like explanatory text or information about the data source. Or make it use 4 columns and 2 rows, or 2 columns and 4 rows so there’s no empty space

Lots of you used facets to show trends over time. By default R tries to make the grid as square as possible, so here there are 3 rows and 3 columns, but that leaves an empty panel in the bottom right corner.

rats_by_year <- rats_clean %>% 
  group_by(borough, sighting_year) %>% 
  summarize(total = n())
## `summarise()` has grouped output by 'borough'. You can override using the
## `.groups` argument.

ggplot(rats_by_year, aes(x = borough, y = total)) +
  geom_col() +
  facet_wrap(vars(sighting_year))

For the sake of balance, you can get rid of that panel by changing the layout. There are 8 panels here, so we could make a rectangle that’s 4 wide and 2 tall (or 2 wide and 4 tall if you want a tall rectangle instead) using the nrow or ncol arguments to facet_wrap():

ggplot(rats_by_year, aes(x = borough, y = total)) +
  geom_col() +
  facet_wrap(vars(sighting_year), ncol = 4)

ggplot(rats_by_year, aes(x = borough, y = total)) +
  geom_col() +
  facet_wrap(vars(sighting_year), nrow = 4)

Alternatively you can stick something in that empty panel like your legend (though in this example it’s better to not even have a legend because it’s redundant with the x-axis). The reposition_legend() function from the {lemon} package makes this really easy:

library(lemon)

p <- ggplot(rats_by_year, aes(x = borough, y = total, fill = borough)) +
  geom_col() +
  facet_wrap(vars(sighting_year)) +
  guides(fill = guide_legend(ncol = 2, title.position = "top"))

reposition_legend(p, position = "bottom left", panel = "panel-3-3")

You can even be fancy and add some explanatory text to that corner. It takes a big of extra work—you essentially have to create a fake text-only plot using grid::textGrob() and then use inset_element() from the {patchwork} to place it on top of the main plot:

library(grid)  # For making custom grid grobs
library(patchwork)

# Make a little text-only plot
extra_note <- textGrob("Here's some text\nabout rats. Neato.", 
                       gp = gpar(fontface = "bold"))

# Run this if you want to see it by itself:
# grid.draw(extra_note)

p <- ggplot(rats_by_year, aes(x = borough, y = total)) +
  geom_col() +
  facet_wrap(vars(sighting_year))

# Add the text-only plot as an inset plot with patchwork
p + inset_element(extra_note, left = 0.7, bottom = 0.0, right = 1, top = 0.3)

Overlapping text

The labels along the x-axis are unreadable and overlapping.

There are lots of ways to fix this—see this whole blog post for some different options. Here are some quick examples (none of these are fabulous, but they’re a start):

ggplot(rats_by_year, aes(x = total, y = borough)) +
  geom_col() +
  facet_wrap(vars(sighting_year), ncol = 4)

ggplot(rats_by_year, aes(x = borough, y = total)) +
  geom_col() +
  facet_wrap(vars(sighting_year), ncol = 4) +
  theme(axis.text.x = element_text(angle = 30, hjust = 0.5, vjust = 0.5))

ggplot(rats_by_year, aes(x = borough, y = total)) +
  geom_col() +
  facet_wrap(vars(sighting_year), ncol = 4) +
  scale_x_discrete(guide = guide_axis(n.dodge = 3))

Commas

Consider adding automatic commas to the x-axis by including library(scales) and adding scale_x_continuous(labels = label_comma())

You can make nicer labels by formatting them with label_comma() (or any of the other label_*() functions) from the {scales} package. See here for a lot more details.

library(scales)

ggplot(rats_summarized, aes(x = borough, y = total)) +
  geom_col() +
  scale_y_continuous(labels = label_comma())

Knit often

Don’t wait until you’ve finished typing your entire document before knitting. Knit to HTML occasionally to make sure all your code runs from a fresh session and to make sure all the extra warnings and messages and any other strange output is taken care of.

Even if your final document won’t be an HTML file, I recommend knitting to HTML often because (1) it’s fast and (2) you can see the output in an easily-refreshable web browser immediately. When you knit to PDF, you have to wait a few extra seconds for LaTeX to compile the rendered Markdown file into PDF, and then you have to reopen the PDF file if you had it open previously. When you knit to Word, if you have a previous version of the .docx file open already, your computer will get mad (especially on Windows), so you have to remember to always close the knitted document in Word before reknitting it, and that’s annoying and you’ll inevitably forget.

So knit to HTML often. Then at the end, once you know everything is working right and the document looks nice and clean, knit to PDF or Word or whatever your final output will be.

Check that your text looks okay

When knitting, you should also check your text to make sure it looks okay. Here are some common issues that you’ll see with your text:

#Task 1: Reflection

# Task 1: Reflection

Some text

## A subheading

Some more text

    It was the best of times, it was the worst of times, it was the age of 
wisdom, it was the age of foolishness, it was the epoch of belief, it was the 
epoch of incredulity, it was the season of Light, it was the season of Darkness, 
it was the spring of hope, it was the winter of despair, we had everything 
before us, we had nothing before us, we were all going direct to Heaven, we were 
all going direct the other way—in short, the period was so far like the present 
period, that some of its noisiest authorities insisted on its being received, 
for good or for evil, in the superlative degree of comparison only.

    There were a king with a large jaw and a queen with a plain face, on the 
throne of England; there were a king with a large jaw and a queen with a fair 
face, on the throne of France. In both countries it was clearer than crystal to 
the lords of the State preserves of loaves and fishes, that things in general 
were settled for ever. 

- Here's a list
- Stuff goes here
- Neato

Don’t install packages in your document

Make sure you don’t include code to install packages in your Rmd files. Like, don’t include install.packages("ggtext") or whatever. If you do, R will reinstall that package every time you knit your document, which is excessive. All you need to do is load the package with library()

To help myself remember to not include package installation code in my document, I make an effort to either install packages with my mouse by clicking on the “Install” button in the Packages panel in RStudio, or only ever typing (or copying/pasting) code like install.packages("whatever") directly in the R console and never putting it in a chunk.

Clean up warnings and messages

Ideally, your document shouldn’t have any errors, warnings, or messages in it. Some packages are talkative, like {tidyverse}:

library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.2     ✔ readr     2.1.4
## ✔ forcats   1.0.0     ✔ stringr   1.5.0
## ✔ ggplot2   3.4.2     ✔ tibble    3.2.1
## ✔ lubridate 1.9.2     ✔ tidyr     1.3.0
## ✔ purrr     1.0.1     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

Some functions are talkative, like summarize() after working on more than one group:

mpg %>% 
  group_by(year, drv) %>% 
  summarize(avg = mean(hwy))
## `summarise()` has grouped output by 'year'. You can override using the
## `.groups` argument.
## # A tibble: 6 × 3
## # Groups:   year [2]
##    year drv     avg
##   <int> <chr> <dbl>
## 1  1999 4      18.8
## 2  1999 f      27.9
## 3  1999 r      20.6
## 4  2008 4      19.5
## 5  2008 f      28.4
## 6  2008 r      21.3

Or geom_smooth():

ggplot(mpg, aes(x = displ, y = hwy)) +
  geom_point() +
  geom_smooth()
## `geom_smooth()` using method = 'loess' and formula = 'y ~ x'

When you have code that you know will return some sort of message, add message=FALSE to the chunk options for it. When you have code that you know will make some sort of warning, (1) try to fix it so that the warning doesn’t happen, or (2) add warning=FALSE to the chunk options for it.

```{r load-packages, warning=FALSE, message=FALSE}
library(tidyverse)
```

Knitting often and scrolling through the rendered document will help you check that you don’t have any extra output like warnings, messages, and errors too.

Global chunk options

You can control the settings for all the chunks in your document by using knitr::opts_chunk$set(). You can use any knitr chunk options (see here or here for a complete list).

For instance, if you want to hide all the code in your document, you could either go add echo=FALSE to each chunk individually, or you could add it to the global chunk options:

knitr::opts_chunk$set(echo = FALSE)

You could hide all warnings and messages there too:

knitr::opts_chunk$set(warning = FALSE, message = FALSE)

BUT I don’t recommend doing that ↑ normally. Warnings are helpful—when you get them, you’ll typically want to try to deal with them. Instead of hiding them, try to fix them. If you can’t fix them (like when you run library(tidyverse)), then you can hide them for that chunk.

You can also set default figure dimensions and lots of other settings. I typically include something like this at the start of my documents:

knitr::opts_chunk$set(
  fig.width = 6, 
  fig.height = 6 * 0.618,  # Golden rectangles!
  fig.align = "center", 
  out.width = "80%"
)

Make sure you hide the chunk that contains those settings with something like include=FALSE, otherwise you’ll see it in your document:

```{r document-chunk-options, include=FALSE}
knitr::opts_chunk$set(
  fig.width = 6, 
  fig.height = 6 * 0.618,  # Golden rectangles!
  fig.align = "center", 
  out.width = "80%"
)
```

Fancy document settings

You can make your documents prettier by changing the settings for different outputs (see this for complete documentation).

For example, if you want a different font in PDFs, you can include something like this in your frontmatter. See here for complete details.

---
title: "Whatever"
date: "Whatever"
output:
  pdf_document:
    latex_engine: xelatex  # This lets LaTeX work with different fonts
mainfont: Garamond
fontsize: 10pt
---

If you want a different template for HTML (you can use these, or if you know HTML and CSS, you can make your own), you can do something like this. See here for complete details.

---
title: "Whatever"
date: "Whatever"
output:
  html_document:
    theme: united
---

You can’t do much with Word because Microsoft makes it hard to do anything to .docx files, but you can create a .docx file where you define a bunch of style settings (e.g., make Heading 1 a specific sans serif font, left-aligned, bold; make Heading 2 the same font and not bold; make main paragraphs an 11pt serif font; and so on), and then you can inject those styles into the knitted document like this. See here for complete details.

---
title: "Whatever"
date: "Whatever"
output:
  word_document:
    reference_docx: my-neat-styles.docx
---

You can keep all those output-specific settings in the same document—knitr will only use the settings required for each output (i.e. if you knit to PDF it’ll ignore the HTML settings):

---
title: "Whatever"
date: "Whatever"
output:
  pdf_document:
    latex_engine: xelatex  # This lets LaTeX work with different fonts
  html_document:
    theme: united
  word_document:
    reference_docx: my-neat-styles.docx
mainfont: Garamond
fontsize: 10pt
---

You can get really fancy with this stuff. For example, this Rmd file creates four different kinds of customized output (see here):

• PDF
• Manuscripty PDF
• HTML
• Word

Quarto

If you want even fancier documents, try the next generation of R Markdown called Quarto. It makes it a billion times easier to do things like change fonts, use cross references (i.e. write things like “see Figure 1” or “see Table 3”), use citations (automatic APA or Chicago citation styles!), use fancier layouts, and so on.

This course website is built with Quarto. Check out this for an example of an academic paper written with Quarto:

• .qmd file
• PDF, manuscripty PDF, and HTML output for that .qmd file

Quarto has great documentation, it’s fully supported in RStudio already, and you basically already know how to use it. It’s just .Rmd with fancier, newer features.

Chunk names

Labeling your R chunks is a good thing to do, since it helps with document navigation and is generally good practice. If you’re using chunk labels make sure you don’t use spaces in them. R will still knit a document with spaceful names, but it converts the spaces to underscores before doing it. So instead of naming chunks like {r My Neat Chunk, message=FALSE}, use something like {r my-neat-chunk} or {r my_neat_chunk}.

In my coefficient plot, the coefficient for per_capita_income didn’t have error bars around it. Why?

(Technically this was from session 7, but I got a lot of questions about it after I posted the FAQs for sessions 7 and 8, so here it is.)

In exercise 7 you created a coefficient plot for a regression model predicting Democratic vote share in the 2016 presidential election based on a bunch of county-level characteristics:

library(tidyverse)
results_2016 <- read_csv("https://datavizs23.classes.andrewheiss.com/files/data/external_data/results_2016.csv")

library(broom)

model <- lm(percent_dem ~ median_age + percent_white + 
              per_capita_income + median_rent + state,
            data = results_2016)

results <- tidy(model, conf.int = TRUE) %>% 
  filter(!str_detect(term, "state"))

results
## # A tibble: 5 × 7
##   term               estimate std.error statistic   p.value  conf.low conf.high
##   <chr>                 <dbl>     <dbl>     <dbl>     <dbl>     <dbl>     <dbl>
## 1 (Intercept)       58.5      1.66          35.3  5.73e-229 55.2      61.8     
## 2 median_age         0.166    0.0368         4.52 6.33e-  6  0.0942    0.238   
## 3 percent_white     -0.682    0.0110       -61.9  0         -0.704    -0.661   
## 4 per_capita_income  0.000271 0.0000497      5.46 5.18e-  8  0.000174  0.000369
## 5 median_rent        0.0200   0.00163       12.3  6.23e- 34  0.0168    0.0232

The point ranges in the plot show the coefficient and the confidence interval around the coefficient. Median rent and per capita income here are just dots, though. Did something go wrong? Why is there no confidence interval?

ggplot(filter(results, term != "(Intercept)"), 
       aes(x = estimate, y = term)) +
  geom_pointrange(aes(xmin = conf.low, xmax = conf.high))

There is a confidence interval, it’s just really tiny! That’s because of how these variables are measured in the data. Each county has a dollar amount recorded for rent and income:

results_2016 %>% 
  select(state, county, per_capita_income, median_rent)
## # A tibble: 3,158 × 4
##    state   county   per_capita_income median_rent
##    <chr>   <chr>                <dbl>       <dbl>
##  1 Alabama Autauga              24571         668
##  2 Alabama Baldwin              26766         693
##  3 Alabama Barbour              16829         382
##  4 Alabama Bibb                 17427         351
##  5 Alabama Blount               20730         403
##  6 Alabama Bullock              18628         276
##  7 Alabama Butler               17403         331
##  8 Alabama Calhoun              20828         422
##  9 Alabama Chambers             19291         374
## 10 Alabama Cherokee             22030         375
## # ℹ 3,148 more rows

When looking at the regression coefficient, we interpret then using single dollars. We’d say something like “A $1 increase in per capita income is associated with a 0.00027 percentage point increase in Democratic vote share, on average.”

Talking about $1 changes, though, is weird because that kind of change in income really isn’t detectable. That’d be like looking at what happens to voting patterns if income in a county goes from $30,000 to $30,001 or $65,000 to $65,001. Nothing’s going to happen to voting patterns! If the original democratic vote share was 60%, a $1 increase in income would increase that by 0.00027 percentage points to 60.00027%.

To make that coefficient more interpretable we can scale down the income column and think of it as hundreds or thousands of dollars instead. We can do that by making a new column in the data…

results_2016 <- results_2016 %>% 
  mutate(per_capita_income_1000 = per_capita_income / 1000)

results_2016 %>% 
  select(state, county, per_capita_income_1000, per_capita_income)
## # A tibble: 3,158 × 4
##    state   county   per_capita_income_1000 per_capita_income
##    <chr>   <chr>                     <dbl>             <dbl>
##  1 Alabama Autauga                    24.6             24571
##  2 Alabama Baldwin                    26.8             26766
##  3 Alabama Barbour                    16.8             16829
##  4 Alabama Bibb                       17.4             17427
##  5 Alabama Blount                     20.7             20730
##  6 Alabama Bullock                    18.6             18628
##  7 Alabama Butler                     17.4             17403
##  8 Alabama Calhoun                    20.8             20828
##  9 Alabama Chambers                   19.3             19291
## 10 Alabama Cherokee                   22.0             22030
## # ℹ 3,148 more rows

…and then using that new per_capita_income_1000 in the model instead:

model <- lm(percent_dem ~ median_age + percent_white + 
              per_capita_income_1000 + median_rent + state,
            data = results_2016)

results <- tidy(model, conf.int = TRUE) %>% 
  filter(!str_detect(term, "state"))

results
## # A tibble: 5 × 7
##   term                 estimate std.error statistic   p.value conf.low conf.high
##   <chr>                   <dbl>     <dbl>     <dbl>     <dbl>    <dbl>     <dbl>
## 1 (Intercept)           58.5      1.66        35.3  5.73e-229  55.2      61.8   
## 2 median_age             0.166    0.0368       4.52 6.33e-  6   0.0942    0.238 
## 3 percent_white         -0.682    0.0110     -61.9  0          -0.704    -0.661 
## 4 per_capita_income_1…   0.271    0.0497       5.46 5.18e-  8   0.174     0.369 
## 5 median_rent            0.0200   0.00163     12.3  6.23e- 34   0.0168    0.0232

Now the coefficient is bigger and we have a more visible confidence interval:

ggplot(filter(results, term != "(Intercept)"), 
       aes(x = estimate, y = term)) +
  geom_pointrange(aes(xmin = conf.low, xmax = conf.high))

Now we can say something like “A $1,000 increase in per capita income is associated with a 0.27 percentage point increase in Democratic vote share, on average” (or going from something like 60% to 60.27%)

The importance of layer order

So far this semester, most of your plots have involved one or two geom_* layers. At one point in some video (I think), I mentioned that layer order doesn’t matter with ggplot. These two chunks of code create identical plots:

ggplot(...) +
  geom_point(...) +
  theme_minimal(...) +
  scale_fill_viridis_c(...) +
  facet_wrap(...) +
  labs(...)

ggplot(...) +
  geom_point(...) +
  labs(...) +
  theme_minimal(...) +
  facet_wrap(...) +
  scale_fill_viridis_c(...)

All those functions can happen in whatever order you want, with one exception. The order of the geom layers matters. The first geom layer you specify will be plotted first, the second will go on top of it, and so on.

Let’s say you want to have a violin plot with jittered points on top. If you put geom_point() first, the points will be hidden by the violins:

library(palmerpenguins)
penguins <- penguins %>% drop_na(sex)

ggplot(penguins, aes(x = species, y = body_mass_g)) +
  geom_point(position = position_jitter(seed = 1234), size = 0.5) +
  geom_violin(aes(fill = species))

To fix it, make sure geom_violin() comes first:

ggplot(penguins, aes(x = species, y = body_mass_g)) +
  geom_violin(aes(fill = species)) +
  geom_point(position = position_jitter(seed = 1234), size = 0.5)

You saw this back in exercise 7 too when you plotted the predicted values of your regression model with geom_line() and a confidence interval with geom_ribbon(). If you put the line first, then the ribbon, the ribbon covers up the line—here that line should be bright red, but the grey of the ribbon makes it darker:

library(marginaleffects)

my_predictions <- predictions(
  model,
  newdata = datagrid(median_age = seq(21, 60, by = 1),
                     state = "Georgia"))

ggplot(my_predictions, aes(x = median_age, y = estimate)) +
  geom_line(linewidth = 1, color = "red") +
  geom_ribbon(aes(ymin = conf.low, ymax = conf.high), alpha = 0.5)

If you put the ribbon first and then the line, the line is the correct shade of red:

ggplot(my_predictions, aes(x = median_age, y = estimate)) +
  geom_ribbon(aes(ymin = conf.low, ymax = conf.high), alpha = 0.5) +
  geom_line(linewidth = 1, color = "red")

This layer order applies to annotation layers too. If you want to highlight an area of the plot (or highlight a recession, like in session 11), adding a rectangle after the geom layers will cover things up, like this ugly yellow rectangle here:

ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g, color = species)) +
  geom_point() +
  annotate(geom = "rect", xmin = 40, xmax = 60, ymin = 5000, ymax = 6100,
           fill = "yellow", alpha = 0.75)

To fix that, put that annotate() layer first, then add other geoms on top:

ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g, color = species)) +
  annotate(geom = "rect", xmin = 40, xmax = 60, ymin = 5000, ymax = 6100,
           fill = "yellow", alpha = 0.75) +
  geom_point()

This doesn’t mean all annotate() layers should come first—if you want an extra label on top of a geom, make sure it comes after:

ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g, color = species)) +
  # Yellow rectangle behind everything
  annotate(geom = "rect", xmin = 40, xmax = 60, ymin = 5000, ymax = 6100,
           fill = "yellow", alpha = 0.75) +
  # Points
  geom_point() +
  # Label on top of the points and the rectangle
  annotate(geom = "label", x = 50, y = 5500, label = "chonky birds")

library(scales)

ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g, color = species)) +
  # Annotations and geoms
  annotate(geom = "rect", xmin = 40, xmax = 60, ymin = 5000, ymax = 6100,
           fill = "yellow", alpha = 0.75) +
  geom_point() +
  annotate(geom = "label", x = 50, y = 5500, label = "chonky birds") +
  # Scale adjustments
  scale_x_continuous(labels = label_comma(scale_cut = cut_si("mm"))) +
  scale_y_continuous(labels = label_comma(scale_cut = cut_si("g"))) +
  scale_color_viridis_d(option = "plasma", end = 0.6) +
  # Guide adjustments
  guides(color = guide_legend(title.position = "left")) +
  # Labels
  labs(x = "Bill length", y = "Body mass", color = "Species:",
       title = "Some title", subtitle = "Penguins!", caption = "Blah") +
  # Facets
  facet_wrap(vars(sex)) +
  # Theme stuff
  theme_minimal() +
  theme(plot.title = element_text(face = "bold", size = rel(1.4)),
        plot.caption = element_text(color = "grey50", hjust = 0),
        axis.title.x = element_text(hjust = 0),
        axis.title.y = element_text(hjust = 1),
        strip.text = element_text(hjust = 0, face = "bold"),
        legend.position = "bottom",
        legend.justification = c(-0.04, 0),
        legend.title = element_text(size = rel(0.9)))

My plot didn’t translate perfectly to ggplotly—why?

In session 10 you used ggplotly() to convert a ggplot object into an interactive plot, which I think is magical:

library(plotly)

basic_plot <- ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g, color = species)) +
  geom_point()

ggplotly(basic_plot)

However, lots of you discovered that Plotly does not translate everything perfectly. Plotly is a separate Javascript library and it doesn’t support every option ggplot does. ggplotly() tries its best to translate between R and Javascript, but it can’t get everything. For instance, subtitles, captions, and labels disappear:

fancy_plot <- ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g, color = species)) +
  geom_point() +
  annotate(geom = "label", x = 50, y = 5500, label = "chonky birds") +
  labs(title = "Penguin bill length and weight",
       subtitle = "Neato", 
       caption = "Here's a caption")

ggplotly(fancy_plot)

That’s just a limitation with ggplot and plotly. If you want a perfect translation, you’ll need to hack into the guts of the translated Javascript and HTML and edit it manually to add those things.

Alternatively, you can check out other interactive plot packages. {ggiraph} makes really great and customizable interactive plots (and it supports things liek subtitles and captions and labels and other annotations ggplotly can’t), but with slightly different syntax:

library(ggiraph)

plot_thing <- ggplot(data = penguins) +
  geom_point_interactive(aes(x = bill_length_mm, y = body_mass_g, color = species,
                             tooltip = species, data_id = species)) +
  annotate(geom = "label", x = 50, y = 5500, label = "chonky birds") +
  labs(title = "Penguin bill length and weight",
       subtitle = "Neato", 
       caption = "Here's a caption")

girafe(ggobj = plot_thing)

Creative Cloud as a student

You can use the whole Creative Cloud Suite for $20/month as a student, and they have a 14-day free trial. Creative Cloud is a subscription service, so you can subscribe on and off as often as you want.

Affinity Suite

Affinity is a smaller rival to Adobe and they have their own set of three programs for graphic design-related tasks. Unlike Creative Cloud, the programs in the Affinity Suite are not subscription-based. You buy them and then you own them forever.

There are three programs that are general equivalents of the core Adobe programs:

• Affinity Photo = Photoshop
• Affinity Designer = Illustrator
• Affinity Publisher = InDesign

Open source alternatives

The open source community has created free programs that are rough equivalents of these core Adobe programs too:

• GIMP = Photoshop
• Inkscape = Illustrator
• Scribus = InDesign

These are all free and they work on macOS and Windows (and Linux if you’re into that), but they can be a little lot rough around the edges and tricky to work with. Adobe, Affinity, and other companies have full time developers focused on making good user interfaces and experiences; these open source clones do not. You can make the same kind of output with GIMP, Inkscape, and Scribus that you can with Adobe Creative Cloud and the Affinity Suite, but there’s a bit of an extra learning curve (and a lot of bumps along the way).

But you can’t beat free.

Summary

Reformat the code

In the R style suggestions in the Resources section, it explains that each layer of a %>%-chained pipeline or ggplot plot should be on separate lines, with the %>% or the + at the end of the line, indented with two spaces.

ggplot(data = blah, mapping = aes(x = thing, y = thing2)) +
  geom_point() +
  geom_smooth(method = "lm") +
  scale_x_continuous() +
  theme_minimal()

Additionally, it’s often a good idea to add lines in between the arguments inside functions and line them up within the ()s of the function.

This makes it so you can clearly see each step of the pipeline or plot, and you can clearly see each of the arguments inside each function.

People tend to take one of two approaches to argument alignment—aligning argument names at the same level as the opening ( of the function like this:

some_object <- some_dataset %>%
  a_function() %>%
  another_function(argument = 1,
                   argument = 2,
                   argument = some_function(thing1 = "a", 
                                            thing2 = "b")) %>%
  yet_another_function()

…or aligning argument names two spaces to the right of where the argument starts, like this:

some_object <- some_dataset %>%
  a_function() %>%
  another_function(
    argument = 1,
    argument = 2,
    argument = some_function(
      thing1 = "a", 
      thing2 = "b"
    )
  ) %>%
  yet_another_function()

RStudio can actually reindent code for you automatically, and it can use either of these approaches. If you want the first approach (where argument names align after the opening (), check “Tools > Global Options > Code > Vertically align arguments in auto-indent”; if you want the second approach (where argument names are all a little indented from where the argument starts), make sure that option is unchecked.

To have RStudio reindent code for you, select the code you want to be reindented and go to “Code > Reindent lines”, or use the keyboard shortcut ⌘I on macOS or ctrl + I on Windows.

Here’s what that looks like. Notice how distorted the indentation is initially—RStudio is smart enough to fix it all:

Not only does reindentation make it easier to read your code, it can reveal issues with the code. Remember that code from the beginning of this post with four things wrong? If we reindent it, the line that starts with is_africa = ifelse( is indented funny—it gets put at the start of the line, when really it should be at the same level as gdp_total, since those are both arguments for the mutate() function. If you look at the line above, you’ll see that there’s a ) after gdpPercap * pop, which closes mutate() prematurely, so is_africa isn’t actually inside mutate(). If we get rid of the ) at the end of pop and reindent again, is_africa shows up in the right place.

I’d recommend trying to keep things indented consistently as you write your code, and periodically reindenting stuff just to make sure everything is nice and aligned. Ultimately R doesn’t care how your code is indented (other languages do, like Python, where one errant space can mess up everything), but humans do care and nicer indentation will help others (and future you!).

Run the code incrementally

Your code is often a series of functions or layers connected with %>% or +. If something goes wrong at any step in the chain of functions, your code won’t work. When that happens, the best strategy for figuring out what went wrong is to run the code incrementally. Just run a few layers of it at a time and then check to see how it looks. Run the first two lines, look at the results, make sure it worked, the run the first three lines, look at the results, make sure it worked, and so on.

This is also a good approach for writing your code initially. That big gapminder-based plot at the beginning of this post? I didn’t write that all at once. I started with the initialy ggplot() + geom_point(), ran it, then added another layer or two, ran it with those, then added some more layers or changed some settings inside existing layers, then ran it with those, and so on until the whole thing was built.

There are a couple ways to do this. One way is to select just the code you want to run (like from the beginning of ggplot() to right before a + on some layer), then press ⌘ + return on macOS or ctrl + enter on Windows to run just that selection. If it worked as expected, select from the beginning again (i.e. at ggplot()) and go to right before a + on some other layer and run that selection:

If you don’t want to keep using your mouse and want to keep your hands at your keyboard, you can add a # right before a + or %>% to comment it out. That essentially breaks the chain of functions at that point, so when you type ⌘ + return or ctrl + enter, R only runs the code up to that point. Then you can remove the #, put it before another + or %>%, and run it again.

Here’s what my typical process for dealing with weirdly indented, broken code looks like. I try to run the whole thing initially, then when it breaks, I reindent it to see if anything is obvious from that. Then I start running it incrementally and check the results of each step to make sure it works up to that point. I do that over and over until the whole pipeline works.

Change the theme

You can reduce the strain on your eyes (and feel more like a stereotypical movie hacker) by changing your theme. I normally use a dark theme (Monokai) since it’s way less bright than the default white theme (Textmate).

If you want to get super fancy, install the {rsthemes} package and you’ll have access to dozens of other neat themes, as well as an add-in for switching between dark and light mode.

Don’t save the workspace when closing

This throws people off a lot. When working with R, datasets you load, plots you create, and variables you make all live in something called a “workspace” or “environment,” which you can see in RStudio’s Environment panel. When you close RStudio, it will ask if you want to save the environment before you leave.

Don’t save it.

Again, don’t save it.

This will feel wrong—you generally always want to save everything all the time, but in this case you don’t. When reopening RStudio, you’ll run code that will load libraries and data and everything else from an empty environment (that’s why you do all your typing in an R Markdown document), and you don’t want to have leftover stuff from previous R sessions in your environment/workspace. (See here for a more in depth explanation.).

You can make RStudio not ask you about saving your environment by making sure the “Restore .RData into workspace at startup” option is unchecked and that “Save workspace to .RData on exit” is set to “Never”:

Turn on rainbow parentheses

You’ll often have nested parentheses…

something(something_else(another_thing()))

…and it can get tricky to keep track of which of those closing )s go with which opening ). To make this easier, you can have RStudio color the parentheses with rotating colors—the first pair will be one color, the next another, and so on.

Turn on color previews

This should theoretically already be enabled by default, but if not, you should definitely turn it on! RStudio can highlight color names with their actual colors. This works for R’s built-in named colors like red and darkblue and for hex colors like #3829ef:

Turn on a code margin

As you can see in the R Style Suggestions page, it’s generally good practice to break up long lines of code. One general standard is to keep your lines at 80 characters (but it’s okay to go beyond that sometimes; there’s no official rule against it). You can turn on a helpful thin gray margin line in the options.

You can set it to whatever width you want in the options:

I knitted my document and nothing happened?

Most of the time this is because R ran into an error when knitting. RStudio shows you knitting errors in the Render panel, but if you’ve minimized that corner of RStudio, or shrunken it down really small, you won’t see it.

Expand the area and look at the panel and you’ll see what went wrong:

If nothing is wrong, you might have set RStudio to not automatically preview the knitted document. Click on the little gear icon at the top of the editor panel and make sure one of the “Preview in…” options is checked:

I tried to knit my document and got an error about duplicate chunk labels. Why?

You can (and should!) name your R code chunks—see here for more about how and why. All chunk names must be unique, though.

Often you’ll copy and paste a chunk from earlier in your document to later, like to make a second plot based on the first. That’s fine—just make sure that you change the chunk name.

If there are chunks with repeated names, R will yell at you:

To fix it, change the name of one of the duplicated names to something unique:

One of my chunks runs just fine in RStudio, but when I knit it doesn’t show anything. Why?

There are lots of different options that you can use with your code chunks (see here for more details). One of these is eval, which controls if the chunk gets run when knitted. By default it’s TRUE, but you can change it to FALSE if you want to include a chunk in your document without actually running it.

This is useful for illustrating things that can’t actually run, like this—there is no function named geom_whatever(), and running ggplot(...) doesn’t do anything, since I use the dots as a placeholder to mean “put stuff in here”:

```{r a-chunk-that-doesnt-run, eval=FALSE}
ggplot(...) +
  geom_whatever()
```

I included a chunk like that in exercise 7:

```{r example-filtering, eval=FALSE}
tidy(...) %>%
  filter(!str_detect(term, "state"))
```

That code doesn’t work by itself because of tidy(...)—again, those dots are just a placeholder. If you make a model and then change the code to use it, that’s great! This code all works:

```{r example-filtering-real, eval=FALSE}
model <- lm(
  percent_dem ~ median_age + percent_white + per_capita_income + median_rent + state,
  data = results_2016
)

tidy(model) %>%
  filter(!str_detect(term, "state"))
```

You can even click on the little play button in RStudio and run the chunk. However, when you knit the document, the code for that chunk will appear in the document, but it won’t run and nothing will appear.

To fix it, either change it to eval=TRUE or just remove that option (it’s true by default)

I tried calculating something with sum() or cor() and R gave me NA instead of a number. Why?

This nearly always happens because of missing values. Let’s make a quick little dataset to illustrate what’s going on (and how to fix it):

library(tidyverse)

example <- tibble(
  x = c(1, 2, 3, 4, 5),
  y = c(6, 7, NA, 9, 10),
  z = c(2, 6, 5, 7, 3)
)

example
## # A tibble: 5 × 3
##       x     y     z
##   <dbl> <dbl> <dbl>
## 1     1     6     2
## 2     2     7     6
## 3     3    NA     5
## 4     4     9     7
## 5     5    10     3

The y column has a missing value (NA), which will mess up any math we do.

Without running any code, what’s the average of the x column? We can find that with math (add all the numbers up and divide by how many numbers there are):

Neat. We can confirm with R:

# With dplyr
example %>% 
  summarize(avg = mean(x))
## # A tibble: 1 × 1
##     avg
##   <dbl>
## 1     3

# With base R
mean(example$x)
## [1] 3

What’s the average of the y column? Math time:

We have no way of knowing what the average is because of that missing value.

If we try it with R, it gives us NA instead of a number:

example %>% 
  summarize(avg = mean(y))
## # A tibble: 1 × 1
##     avg
##   <dbl>
## 1    NA

To fix this, we can tell R to remove all the missing values from the column before calculating the average so that it does this:

Include the argument na.rm = TRUE to do that:

example %>% 
  summarize(avg = mean(y, na.rm = TRUE))
## # A tibble: 1 × 1
##     avg
##   <dbl>
## 1     8

This works for lots of R’s calculating functions, like sum(), min(), max(), sd(), median(), mean(), and so on:

example %>% 
  summarize(
    total = sum(y, na.rm = TRUE),
    avg = mean(y, na.rm = TRUE),
    median = median(y, na.rm = TRUE),
    min = min(y, na.rm = TRUE),
    max = max(y, na.rm = TRUE),
    std_dev = sd(y, na.rm = TRUE)
  )
## # A tibble: 1 × 6
##   total   avg median   min   max std_dev
##   <dbl> <dbl>  <dbl> <dbl> <dbl>   <dbl>
## 1    32     8      8     6    10    1.83

This works a little differently with cor() because you’re working with multiple columns instead of just one. If there are any missing values in any of the columns you’re correlating, you’ll get NA for the columns that use it. Here, we have a correlation between x and z because there are no missing values in either of those, but we get NA for the correlation between x and y and between z and y:

example %>% 
  cor()
##           x  y         z
## x 1.0000000 NA 0.2287479
## y        NA  1        NA
## z 0.2287479 NA 1.0000000

Adding na.rm to cor() doesn’t work because cor() doesn’t actually have an argument for na.rm:

example %>%
  cor(na.rm = TRUE)
## Error in cor(., na.rm = TRUE): unused argument (na.rm = TRUE)

Instead, if you look at the documentation for cor() (run ?cor in your R console or search for it in the Help panel in RStudio), you’ll see an argument named use instead. By default it will use all the rows in the data (use = "everything"), but we can change it to use = "complete.obs". This will remove all rows where something is missing before calculating the correlation:

example %>%
  cor(use = "complete.obs")
##           x         y         z
## x 1.0000000 1.0000000 0.2300895
## y 1.0000000 1.0000000 0.2300895
## z 0.2300895 0.2300895 1.0000000

I want my bars to be sorted in my plot. How can I control their order?

Sorting categories by different values is important for showing trends in your data. By default, R will plot categorical variables in alphabetical order, but often you’ll want these categories to use some sort of numeric order, likely based on a different column.

There are a few different ways to sort categories. First, let’s make a summarized dataset of the total population in each continent in 2007 (using our trusty ol’ gapminder data):

library(gapminder)

# Find the total population in each continent in 2007
population_by_continent <- gapminder %>% 
  filter(year == 2007) %>% 
  group_by(continent) %>% 
  summarize(total_population = sum(pop))
population_by_continent
## # A tibble: 5 × 2
##   continent total_population
##   <fct>                <dbl>
## 1 Africa           929539692
## 2 Americas         898871184
## 3 Asia            3811953827
## 4 Europe           586098529
## 5 Oceania           24549947

By default the continents will be in alphabetic order:

ggplot(
  population_by_continent,
  aes(x = continent, y = total_population, fill = continent)
) +
  geom_col() +
  # The legend is redundant because of the x-axis
  guides(fill = "none")

In this case it’s more useful to plot these in order of total population. My favorite approach for this is to (1) sort the data how I want it with arrange() and (2) lock the order of the category in place with fct_inorder(). Note how the mini dataset is now sorted and Oceania comes first:

plot_data_sorted <- population_by_continent %>% 
  # Sort by population
  arrange(total_population) %>% 
  # Make continent use the order it's currently in
  mutate(continent = fct_inorder(continent))
plot_data_sorted
## # A tibble: 5 × 2
##   continent total_population
##   <fct>                <dbl>
## 1 Oceania           24549947
## 2 Europe           586098529
## 3 Americas         898871184
## 4 Africa           929539692
## 5 Asia            3811953827

If we plot it, the continents will be in order:

ggplot(
  plot_data_sorted,
  aes(x = continent, y = total_population, fill = continent)
) +
  geom_col() +
  guides(fill = "none")

This plots the continents in reverse order, with Oceania on the left. We can reverse this by either arranging the data in descending population order, or by using fct_rev() to reverse the continent order:

plot_data_sorted <- population_by_continent %>% 
  # Sort by population in descending order
  arrange(desc(total_population)) %>% 
  # Lock in the continent order
  mutate(continent = fct_inorder(continent))

ggplot(
  plot_data_sorted,
  aes(x = continent, y = total_population, fill = continent)
) +
  geom_col() +
  guides(fill = "none")

plot_data_sorted <- population_by_continent %>% 
  # Sort by population in ascending order
  arrange(total_population) %>% 
  # Lock in the continent order
  mutate(continent = fct_inorder(continent))

ggplot(
  plot_data_sorted,
  # Reverse the continent order with fct_rev()
  aes(x = fct_rev(continent), y = total_population, fill = fct_rev(continent))
) +
  geom_col() +
  guides(fill = "none")

An alternative to the two-step arrange() %>% mutate(blah = fct_inorder(blah)) is to use fct_reorder(), which takes two arguments: (1) the column you want to be reordered and (2) the column you want to sort it by:

plot_data_sorted <- population_by_continent %>% 
  # Sort continent by total_population in descending order
  mutate(continent = fct_reorder(continent, total_population, .desc = TRUE))

ggplot(
  plot_data_sorted,
  aes(x = continent, y = total_population, fill = continent)
) +
  geom_col() +
  guides(fill = "none")

That’s only one line instead of two, which is nice, but I tend to be fan of the two step approach because it’s more explicit and gives me more control over sorting. For instance, here I want all the gapminder countries to be sorted by year (descending), continent, and life expectancy so we can see descending life expectancy within each continent over time.

I’m sure there’s a way to sort by multiple columns in different orders like this with fct_reorder(), but I don’t know how. Plus, if I run this super_sorted_data code up until the end of arrange(), I can look at it in RStudio to make sure all the ordering I want is right. That’s harder to do with fct_reorder().

super_sorted_data <- gapminder %>% 
  filter(year >= 1997) %>% 
  # Get the countries in order of year (descending), continent, and life expectancy
  arrange(desc(year), continent, lifeExp) %>%
  # Lock the country name order in place + lock the year in place
  mutate(
    country = fct_inorder(country),
    # year is currently a number, so we need to change it to a factor before
    # reordering it
    year = fct_inorder(factor(year))
  )

ggplot(super_sorted_data, aes(y = country, x = lifeExp, fill = continent)) +
  geom_col() +
  facet_wrap(vars(year)) +
  # Reverse the order legend so that Oceania is at the top, since it's at the
  # top in the plot
  guides(fill = guide_legend(reverse = TRUE)) +
  theme_minimal() +
  # Remove country names and y-axis gridlines + put legend on the bottom
  theme(
    axis.text.y = element_blank(),
    axis.ticks.y = element_blank(),
    panel.grid.major.y = element_blank(),
    panel.grid.minor.y = element_blank(),
    legend.position = "bottom"
  )

You can also specify any arbitrary category order with fct_relevel()

plot_data_sorted <- population_by_continent %>% 
  # Use this specific continent order
  mutate(continent = fct_relevel(
    continent, 
    c("Asia", "Oceania", "Europe", "Americas", "Africa"))
  )

ggplot(
  plot_data_sorted,
  aes(x = continent, y = total_population, fill = continent)
) +
  geom_col() +
  guides(fill = "none")

My data has full US state names but I want to use abbreviations (or regions). Is there a way to automatically convert from names to something else?

Yep! R has a few state-related variables built in (run ?state in your R console to see them all):

state.name
##  [1] "Alabama"        "Alaska"         "Arizona"        "Arkansas"      
##  [5] "California"     "Colorado"       "Connecticut"    "Delaware"      
##  [9] "Florida"        "Georgia"        "Hawaii"         "Idaho"         
## [13] "Illinois"       "Indiana"        "Iowa"           "Kansas"        
## [17] "Kentucky"       "Louisiana"      "Maine"          "Maryland"      
## [21] "Massachusetts"  "Michigan"       "Minnesota"      "Mississippi"   
## [25] "Missouri"       "Montana"        "Nebraska"       "Nevada"        
## [29] "New Hampshire"  "New Jersey"     "New Mexico"     "New York"      
## [33] "North Carolina" "North Dakota"   "Ohio"           "Oklahoma"      
## [37] "Oregon"         "Pennsylvania"   "Rhode Island"   "South Carolina"
## [41] "South Dakota"   "Tennessee"      "Texas"          "Utah"          
## [45] "Vermont"        "Virginia"       "Washington"     "West Virginia" 
## [49] "Wisconsin"      "Wyoming"
state.abb
##  [1] "AL" "AK" "AZ" "AR" "CA" "CO" "CT" "DE" "FL" "GA" "HI" "ID" "IL" "IN" "IA"
## [16] "KS" "KY" "LA" "ME" "MD" "MA" "MI" "MN" "MS" "MO" "MT" "NE" "NV" "NH" "NJ"
## [31] "NM" "NY" "NC" "ND" "OH" "OK" "OR" "PA" "RI" "SC" "SD" "TN" "TX" "UT" "VT"
## [46] "VA" "WA" "WV" "WI" "WY"
state.region
##  [1] South         West          West          South         West         
##  [6] West          Northeast     South         South         South        
## [11] West          West          North Central North Central North Central
## [16] North Central South         South         Northeast     South        
## [21] Northeast     North Central North Central South         North Central
## [26] West          North Central West          Northeast     Northeast    
## [31] West          Northeast     South         North Central North Central
## [36] South         West          Northeast     Northeast     South        
## [41] North Central South         South         West          Northeast    
## [46] South         West          South         North Central West         
## Levels: Northeast South North Central West

These aren’t datasets—they’re single vectors—but you can make a little dataset with columns for each of those details, like this:

state_details <- tibble(
  state = state.name,
  state_abb = state.abb,
  state_division = state.division,
  state_region = state.region
) %>% 
  # Add DC manually
  add_row(
    state = "Washington, DC",
    state_abb = "DC",
    state_division = "South Atlantic",
    state_region = "South"
  )
state_details
## # A tibble: 51 × 4
##    state       state_abb state_division     state_region
##    <chr>       <chr>     <chr>              <chr>       
##  1 Alabama     AL        East South Central South       
##  2 Alaska      AK        Pacific            West        
##  3 Arizona     AZ        Mountain           West        
##  4 Arkansas    AR        West South Central South       
##  5 California  CA        Pacific            West        
##  6 Colorado    CO        Mountain           West        
##  7 Connecticut CT        New England        Northeast   
##  8 Delaware    DE        South Atlantic     South       
##  9 Florida     FL        South Atlantic     South       
## 10 Georgia     GA        South Atlantic     South       
## # ℹ 41 more rows

You can join this dataset to any data you have that has state names or state abbreviations. Joining the data will bring all the columns from state_details into your data wherever rows match. You’ll learn a lot more about joining things in sesison 12 too.

For instance, imagine you have a dataset that looks like this, similar to the unemployment data from exercise 8:

some_state_data <- tribble(
  ~state, ~something,
  "Wyoming", 5,
  "North Carolina", 9,
  "Nevada", 10,
  "Georgia", 3,
  "Rhode Island", 1,
  "Washington, DC", 6
)
some_state_data
## # A tibble: 6 × 2
##   state          something
##   <chr>              <dbl>
## 1 Wyoming                5
## 2 North Carolina         9
## 3 Nevada                10
## 4 Georgia                3
## 5 Rhode Island           1
## 6 Washington, DC         6

We can merge in (or join) the state_details data so that we add columns for abbreviation, region, and so on, using left_join() (again, see lesson 12 for more about all this):

# Join the state details
data_with_state_details <- some_state_data %>% 
  left_join(state_details, by = join_by(state))
data_with_state_details
## # A tibble: 6 × 5
##   state          something state_abb state_division state_region
##   <chr>              <dbl> <chr>     <chr>          <chr>       
## 1 Wyoming                5 WY        Mountain       West        
## 2 North Carolina         9 NC        South Atlantic South       
## 3 Nevada                10 NV        Mountain       West        
## 4 Georgia                3 GA        South Atlantic South       
## 5 Rhode Island           1 RI        New England    Northeast   
## 6 Washington, DC         6 DC        South Atlantic South

Now your data_with_state_details data has new columns for abbreviations, divisions, regions, and everything else that was in state_details:

# Use it
ggplot(
  data_with_state_details, 
  aes(x = state_abb, y = something, fill = state_region)
) +
  geom_col()

And for fun, we can fix the ordering:

# Fix the ordering
data_with_state_details <- some_state_data %>% 
  left_join(state_details, by = join_by(state)) %>% 
  arrange(desc(something)) %>% 
  mutate(state_abb = fct_inorder(state_abb))

ggplot(
  data_with_state_details, 
  aes(x = state_abb, y = something, fill = state_region)
) +
  geom_col()

My axis labels are overlapping and ugly. How can I fix that?

Sometimes you’ll have text that is too long to fit comfortably as axis labels and the labels will overlap and be gross:

Check out this blog post for a bunch of different ways to fix this and make nicer labels, like this:

Why isn’t the example code using data = whatever and mapping = aes() in ggplot() anymore? Do we not have to use argument names?

In the first few sessions, you wrote code that looked like this:

ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
  geom_point()

In R, you feed functions arguments like data and mapping and I was having you explicitly name the arguments, like data = mpg and mapping = aes(...).

In general it’s a good idea to use named arguments, since it’s clearer what you mean.

However, with really common functions like ggplot(), you can actually skip the names. If you look at the documentation for ggplot() (i.e. run ?ggplot in your R console or search for “ggplot” in the Help panel in RStudio), you’ll see that the first expected argument is data and the second is mapping.

If you don’t name the arguments, like this…

ggplot(mpg, aes(x = displ, y = hwy)) +
  geom_point()

…ggplot() will assume that the first argument (mpg) really means data = mpg and that the second really means mapping = aes(...).

If you don’t name the arguments, the order matters. This won’t work because ggplot will think that the aes(...) stuff is really data = aes(...):

ggplot(aes(x = displ, y = hwy), mpg) +
  geom_point()

If you do name the arguments, the order doesn’t matter. This will work because it’s clear that data = mpg (even though this feels backwards and wrong):

ggplot(mapping = aes(x = displ, y = hwy), data = mpg) +
  geom_point()

This works with all R functions. You can either name the arguments and put them in whatever order you want, or you can not name them and use them in the order that’s listed in the documentation.

In general, you should name your arguments for the sake of clarity. For instance, with aes(), the first argument is x and the second is y, so you can technically do this:

ggplot(mpg, aes(displ, hwy)) +
  geom_point()

That’s nice and short, but you have to remember that displ is on the x-axis and hwy is on the y-axis. And it gets extra confusing once you start mapping other columns:

ggplot(mpg, aes(displ, hwy, color = drv, size = hwy)) +
  geom_point()

All the other aesthetics like color and size are named, but x and y aren’t, which just feels… off.

So use argument names except for super common things like ggplot() and the {dplyr} verbs like mutate(), group_by(), filter(), etc.

In chapter 22, Wilke talks about tables—is there a way to make pretty tables with R?

Absolutely! We don’t have time in this class to cover tables, but there’s a whole world of packages for making beautiful tables with R. Three of the more common ones are {gt}, {kableExtra}, and {flextable}:

Here’s a quick illustration of these three packages. All three are incredibly powerful and let you do all sorts of really neat formatting things ({gt} even makes interactive HTML tables!), so make sure you check out the documentation and examples. I personally use all three, depending on which output I’m working with. When knitting to HTML, I use {gt}; when knitting to PDF I use {gt} or {kableExtra}; when knitting to Word I use {flextable}.

library(tidyverse)

cars_summary <- mpg %>% 
  group_by(year, drv) %>%
  summarize(
    n = n(),
    avg_mpg = mean(hwy),
    median_mpg = median(hwy),
    min_mpg = min(hwy),
    max_mpg = max(hwy)
  )

library(gt)

cars_summary %>% 
  gt() %>% 
  cols_label(
    drv = "Drive",
    n = "N",
    avg_mpg = "Average",
    median_mpg = "Median",
    min_mpg = "Minimum",
    max_mpg = "Maximum"
  ) %>% 
  tab_spanner(
    label = "Highway MPG",
    columns = c(avg_mpg, median_mpg, min_mpg, max_mpg)
  ) %>% 
  fmt_number(
    columns = avg_mpg,
    decimals = 2
  ) %>% 
  tab_options(
    row_group.as_column = TRUE
  )

library(kableExtra)

cars_summary %>% 
  ungroup() %>% 
  select(-year) %>% 
  kbl(
    col.names = c("Drive", "N", "Average", "Median", "Minimum", "Maximum"),
    digits = 2
  ) %>% 
  kable_styling() %>% 
  pack_rows("1999", 1, 3) %>% 
  pack_rows("2008", 4, 6) %>% 
  add_header_above(c(" " = 2, "Highway MPG" = 4))

library(flextable)

cars_summary %>% 
  rename(
    "Year" = year,
    "Drive" = drv,
    "N" = n,
    "Average" = avg_mpg,
    "Median" = median_mpg,
    "Minimum" = min_mpg,
    "Maximum" = max_mpg
    ) %>% 
  mutate(Year = as.character(Year)) %>% 
  flextable() %>% 
  colformat_double(j = "Average", digits = 2) %>%
  add_header_row(values = c(" ", "Highway MPG"), colwidths = c(3, 4)) %>% 
  align(i = 1, part = "header", align = "center") %>% 
  merge_v(j = ~ Year) %>% 
  valign(j = 1, valign = "top")

You can also create more specialized tables for specific situations, like side-by-side regression results tables with {modelsummary} (which uses {gt}, {kableExtra}, or {flextable} behind the scenes)

library(modelsummary)

model1 <- lm(hwy ~ displ, data = mpg)
model2 <- lm(hwy ~ displ + drv, data = mpg)

modelsummary(
  list(model1, model2),
  stars = TRUE,
  # Rename the coefficients
  coef_rename = c(
    "(Intercept)" = "Intercept",
    "displ" = "Displacement",
    "drvf" = "Drive (front)",
    "drvr" = "Drive (rear)"),
  # Get rid of some of the extra goodness-of-fit statistics
  gof_omit = "IC|RMSE|F|Log",
  # Use {gt}
  output = "gt"
)

Double encoding and excessive legends

As you’ve read, double encoding aesthetics can be helpful for accessibility and printing reasons—for instance, if points have colors and shapes, they’re still readable by people who are colorblind or if the image is printed in black and white:

ggplot(mpg, aes(x = displ, y = hwy, color = drv, shape = drv)) +
  geom_point()

Sometimes the double encoding can be excessive though, and you can safely remove legends. For example, in exercises 3 and 4, you made bar charts showing counts of different things (words spoken in The Lord of the Rings; pandemic-era construction projects in New York City), and lots of you colored the bars, which is great!

car_counts <- mpg %>% 
  group_by(drv) %>% 
  summarize(n = n())

ggplot(car_counts, aes(x = drv, y = n, fill = drv)) +
  geom_col()

Car drive here is double encoded: it’s on the x-axis and it’s the fill. That’s great, but having the legend here is actually a little excessive. Both the x-axis and the legend tell us what the different colors of drives are (four-, front-, and rear-wheeled drives), so we can safely remove the legend and get a little more space in the plot area:

ggplot(car_counts, aes(x = drv, y = n, fill = drv)) +
  geom_col() +
  guides(fill = "none")

I have numbers like 20000 and want them formatted with commas like 20,000. Can I do that automatically?

Yes you can! There’s an incredible package called {scales}. It lets you format numbers and axes and all sorts of things in magical ways. If you look at the documentation, you’ll see a ton of label_SOMETHING() functions, like label_comma(), label_dollar(), and label_percent().

You can use these different labeling functions inside scale_AESTHETIC_WHATEVER() layers in ggplot.

label_comma() adds commas:

library(scales)
library(gapminder)

gapminder_2007 <- gapminder %>% 
  filter(year == 2007)

ggplot(gapminder_2007, aes(x = gdpPercap)) +
  geom_histogram(binwidth = 1000) +
  scale_x_continuous(labels = label_comma())

label_dollar() adds commas and includes a “$” prefix:

ggplot(gapminder_2007, aes(x = gdpPercap)) +
  geom_histogram(binwidth = 1000) +
  scale_x_continuous(labels = label_dollar())

label_percent() multiplies values by 100 and formats them as percents:

gapminder_percents <- gapminder_2007 %>% 
  group_by(continent) %>% 
  summarize(n = n()) %>% 
  mutate(prop = n / sum(n))

ggplot(gapminder_percents, aes(x = continent, y = prop)) +
  geom_col() +
  scale_y_continuous(labels = label_percent())

You can also change a ton of the settings for these different labeling functions. Want to format something as Euros and use periods as the number separators instead of commas, like Europeans? Change the appropriate arguments! You can check the documentation for each of the label_WHATEVER() functions to see what you can adjust (like label_dollar() here)

ggplot(gapminder_2007, aes(x = gdpPercap)) +
  geom_histogram(binwidth = 1000) +
  scale_x_continuous(labels = label_dollar(prefix = "€", big.mark = "."))

All the label_WHATEVER() functions actually create copies of themselves, so if you’re using lots of custom settings, you can create your own label function, like label_euro() here:

# Make a custom labeling function
label_euro <- label_dollar(prefix = "€", big.mark = ".")

# Use it on the x-axis
ggplot(gapminder_2007, aes(x = gdpPercap)) +
  geom_histogram(binwidth = 1000) +
  scale_x_continuous(labels = label_euro)

These labeling functions also work with other aesthetics, like fill and color and size. Use them in scale_AESTHETIC_WHATEVER():

ggplot(
  gapminder_2007, 
  aes(x = gdpPercap, y = lifeExp, size = pop, color = pop)
) +
  geom_point() +
  scale_x_continuous(labels = label_dollar()) +
  scale_size_continuous(labels = label_comma()) +
  scale_color_viridis_c(labels = label_comma())

There are also some really neat and fancy things you can do with scales, like formatting logged values, abbreviating long numbers, and many other things. Check out this post for an example of working with logged values.

ggplot(
  gapminder_2007, 
  aes(x = gdpPercap, y = lifeExp, size = pop, color = continent)
) +
  geom_point() +
  scale_x_log10(breaks = 500 * 2^seq(0, 9, by = 1),
                labels = label_dollar(scale_cut = cut_short_scale())) +
  scale_size_continuous(labels = label_comma(scale_cut = cut_short_scale()))

Legends are cool, but I’ve read that directly labeling things can be better. Is there a way to label things without a legend?

Yes! Later in the semester we’ll cover annotations, but in the meantime, you can check out a couple packages that let you directly label geoms that have been mapped to aesthetics.

# This isn't on CRAN, so you need to install it by running this:
# remotes::install_github("AllanCameron/geomtextpath")
library(geomtextpath)
library(palmerpenguins)  # Penguin data

# Get rid of the rows that are missing sex
penguins <- penguins %>% drop_na(sex)

ggplot(
  penguins, 
  aes(x = bill_length_mm, y = body_mass_g, color = species)
) +
  geom_point(alpha = 0.5) +  # Make the points a little bit transparent
  geom_labelsmooth(
    aes(label = species), 
    # This spreads the letters out a bit
    text_smoothing = 80
  ) +
  # Turn off the legend bc we don't need it now
  guides(color = "none")

gapminder_lifeexp <- gapminder %>% 
  group_by(continent, year) %>% 
  summarize(avg_lifeexp = mean(lifeExp))

ggplot(
  gapminder_lifeexp, 
  aes(x = year, y = avg_lifeexp, color = continent)
) +
  geom_textline(
    aes(label = continent, hjust = continent),
    linewidth = 1, size = 4
  ) +
  guides(color = "none")

# This also isn't on CRAN, so you need to install it by running this:
# remotes::install_github("MattCowgill/ggdirectlabel")
library(ggdirectlabel)

ggplot(
  penguins, 
  aes(x = bill_length_mm, y = body_mass_g, color = species)
) +
  geom_point(alpha = 0.5) +
  geom_smooth() +
  geom_richlegend(
    aes(label = species),  # Use the species as the fake legend labels
    legend.position = "topleft",  # Put it in the top left
    hjust = 0  # Make the text left-aligned (horizontal adjustment, or hjust)
  ) +
  guides(color = "none")

Talking about packages and functions

You’ve probably noticed that on the course website here, I put package names in {}s, like {ggplot2} or {gghalves}. This is a normal convention in the R world—people generally either put package names in braces or in bold.

When writing about functions, I typically format them as code, followed by empty open and closed parentheses, like geom_point(). That signals that it’s actual runnable code and not the name of a package. Remember that you can format things like inline code by using single backticks.

For instance, this:

You can use `geom_point()` from {ggplot2} to make scatterplots.

will knit into this:

• You can use geom_point() from {ggplot2} to make scatterplots.

Nicer {ggplot2} documentation

The ggplot documentation within R (i.e. in the help panel) is good, but I find that it’s nicer to use the documentation website. It’s the same exact content, but the website version shows plots for each of the examples. Scroll down to the examples section for geom_point(), for instance.

Installing vs. using packages

Make sure you don’t include code to install packages in your Rmd files. Like, don’t include install.packages("gapminder") or whatever. If you do, R will reinstall that package every time you knit your document, which is excessive. All you need to do is load the package with library().

# Only do this once on your computer! Don't include this line of code in a document
install.packages("gapminder")

# This loads the gapminder package and lets you use it
library(gapminder)

The tidyverse package shortcut

When you run library(tidyverse), R shows a message that says it has loaded 9 packages:

── Attaching core tidyverse packages ────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.2     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ stringr   1.5.0
✔ ggplot2   3.4.2     ✔ tibble    3.2.1
✔ lubridate 1.9.2     ✔ tidyr     1.3.0
✔ purrr     1.0.1     

The tidyverse package developers have found that those 9 are some of the most common packages that people use, so they created library(tidyverse) as a shortcut for loading them all at the same time. Alternatively, you could start your documents like this:

library(ggplot2)
library(dplyr)
library(readr)
library(tidyr)
library(stringr)
library(forcats)
# etc.

But that’s a lot of typing. It’s easier to just do library(tidyverse)

If you load the tidyverse package, you don’t need to load those 9 individual packages. Doing this is entirely redundant:

library(tidyverse)
library(ggplot2)  # This is alreaady loaded bc tidyverse
library(dplyr)  # This is also already loaded bc tidyverse

I’ve tried to point out in your exercises if/when you do this

Chunk labels

Labeling your R chunks is a good thing to do, since it helps with document navigation and is generally good practice. If you’re using chunk labels make sure you don’t use spaces in them. R will still knit a document with spaceful names, but it converts the spaces to underscores before doing it. So instead of naming chunks like {r My Neat Chunk, message=FALSE}, use something like {r my-neat-chunk} or {r my_neat_chunk}.

Code style

Unlike other programming languages (grumbles at Python), R is fairly forgiving with the style of the code you write. You can have extra spaces, you can omit spaces, you can indent things however you want, etc.

But if you want to write readable code (you do!) and write code that others can work with (you do!), you should follow some basic style guidelines. I’ve summarized a few of the most important ones here.

I’m never going to grade you on any of this, by the way! These are just a set of best practices that you should get into the habit of using.

Why are my axis labels all crowded and on top of each other? How do I fix that?

This was a common problem with both the LOTR data and the essential construction data—categories on the x-axis would often overlap when you knit your document. That’s because there’s not enough room to fit them all comfortably in a standard image size. Fortunately there are a few quick and easy ways to fix this, such as changing the width of the image (see below), rotating the labels, dodging the labels, or (my favorite!) automatically adding line breaks to the labels so they don’t overlap. This blog post (by me) has super quick examples of all these different (easy!) approaches.

How can I add a line break to the text in my plot?

If you don’t want to use the fancier techniques from the blog post about long labels, a quick and easy way to deal with longer text is to manually insert a linebreak yourself. This is super easy: include a \n where you want a new line:

ggplot(data = mpg, mapping = aes(x = displ, y = hwy, color = drv)) +
  geom_point() +
  labs(
    title = "Everyone's favorite\nexample plot",
    x = "Displacement\n(proxy for car weight)",
    y = "MPG\n(highway)",
    color = "Drive"
  )

How can I control the dimensions of my figures?

By default, R creates plots that are 7″×7″ squares:

library(tidyverse)

ggplot(data = mpg, mapping = aes(x = displ, y = hwy, color = drv)) +
  geom_point()

Often, though, those plots are excessively large and can result in text that is too small and dimensions that feel off. You generally want to have better control over the dimensions of the figures you make. For instance, you can make them landscape when there’s lots of text involved. To do this, you can use the fig.width and fig.height chunk options to control the, um, width and height of the figures:

```{r landscape-plot, fig.width=5, fig.height=3}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy, color = drv)) +
  geom_point()
```

The dimensions are also reflected in RStudio itself when you’re working with inline images, so it’s easy to tinker with different values and rerun the chunk without needing to re-knit the whole document over and over again:

```{r landscape-plot-golden, fig.width=6, fig.height=(6 / 1.618)}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy, color = drv)) +
  geom_point()
```

What’s the difference between read_csv() vs. read.csv()?

In all the code I’ve given you in this class, you’ve loaded CSV files using read_csv(), with an underscore. In lots of online examples of R code, and in lots of other peoples’ code, you’ll see read.csv() with a period. They both load CSV files into R, but there are subtle differences between them.

read.csv() (read dot csv) is a core part of R and requires no external pacakges (we say that it’s part of “base R”). It loads CSV files. That’s its job. However, it can be slow with big files, and it can sometimes read text data in as categorical data, which is weird (that’s less of an issue since R 4.0; it was a major headache in the days before R 4.0). It also makes ugly column names when there are “illegal” columns in the CSV file—it replaces all the illegal characters with .s

mpg %>% 
  group_by(drv) %>% 
  summarize(`A column with spaces` = mean(hwy))
## # A tibble: 3 × 2
##   drv   `A column with spaces`
##   <chr>                  <dbl>
## 1 4                       19.2
## 2 f                       28.2
## 3 r                       21

read_csv() (read underscore csv) comes from {readr}, which is one of the 9 packages that get loaded when you run library(tidyverse). Think of it as a new and improved version of read.csv(). It handles big files a better, it doesn’t ever read text data in as categorical data, and it does a better job at figuring out what kinds of columns are which (if it detects something that looks like a date, it’ll treat it as a date). It also doesn’t rename any columns—if there are illegal characters like spaces, it’ll keep them for you, which is nice.

Moral of the story: use read_csv() instead of read.csv(). It’s nicer.

Why does R keep yelling at me with warnings and messages?

By now you’ve seen ominous looking red text in R, like 'summarise()' has grouped output by 'Gender'. You can override using the '.groups' argument or Warning: Removed 2 rows containing missing values, and so on. You might have panicked a little after seeing this and thought you were doing something wrong.

Never fear! You’re most likely not doing anything wrong.

R shows red text in the console pane in three different situations:

• Errors: When the red text is a legitimate error, it will be prefaced with “Error in…” and will try to explain what went wrong. Generally when there’s an error, the code will not run. For example, if you see Error in ggplot(...) : could not find function "ggplot", it means that the ggplot() function is not accessible because the package that contains the function (ggplot2) was not loaded with library(ggplot2) (or library(tidyverse), which loads ggplot2). Thus you cannot use the ggplot() function without the ggplot2 package being loaded first.
• Warnings: When the red text is a warning, it will be prefaced with “Warning:” and R will try to explain why there’s a warning. Generally your code will still work, but with some caveats. For example if you create a scatterplot based on a dataset where two of the rows of data have missing entries that would be needed to create points in the scatterplot, you will see this warning: Warning: Removed 2 rows containing missing values (geom_point). R will still produce the scatterplot with all the remaining non-missing values, but it is warning you that two of the points aren’t there.
• Messages: When the red text doesn’t start with either “Error” or “Warning”, it’s just a friendly message. You’ll see these messages when you load R packages or when you read data saved in spreadsheet files with read_csv(). These are helpful diagnostic messages and they don’t stop your code from working. This is what 'summarise()' has grouped output by 'Gender'... is—just a helpful note.

Remember, when you see red text in the console, don’t panic. It doesn’t necessarily mean anything is wrong. Rather:

• If the text starts with “Error”, figure out what’s causing it. Think of errors as a red traffic light: something is wrong!
• If the text starts with “Warning”, figure out if it’s something to worry about. For instance, if you get a warning about missing values in a scatterplot and you know there are missing values, you’re fine. If that’s surprising, look at your data and see what’s missing. Think of warnings as a yellow traffic light: everything is working fine, but watch out/pay attention.
• Otherwise, the text is just a message. Read it, wave back at R, and thank it for talking to you. Think of messages as a green traffic light: everything is working fine and keep on going!

How can I hide warnings and messages?

In general, you’ll want to try to deal with errors and warnings, often by adjusting or clarifying something in your code. In your final knitted documents, you typically want to have nice clean output without any warnings or messages. You can fix these warnings and messages in a couple ways: (1) change your code to deal with them, or (2) just hide them.

For instance, if you do something like this to turn off the fill legend:

# Not actual code; don't try to run this
ggplot(data = whatever, aes(x = blah, y = blah, fill = blah)) +
  geom_col() +
  guides(fill = FALSE)

You’ll get this warning:

## Warning: The `<scale>` argument of `guides()` cannot be `FALSE`. Use "none" 
## instead as of ggplot2 3.3.4.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

You’ll still get a plot and the fill legend will be gone and that’s great, but the warning is telling you that that code has been deprecated and is getting phased out and will eventually stop working. ggplot helpfully tells you how to fix it: use guides(fill = "none") instead. Changing that code removes the warning and everything will work just fine:

# Not actual code; don't try to run this
ggplot(data = whatever, aes(x = blah, y = blah, fill = blah)) +
  geom_col() +
  guides(fill = "none")

In other cases, though, nothing’s wrong and R is just being talkative. For instance, when you load {tidyverse}, you get a big wall of text:

library(tidyverse)
## ── Attaching core tidyverse packages ─────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.2     ✔ readr     2.1.4
## ✔ forcats   1.0.0     ✔ stringr   1.5.0
## ✔ ggplot2   3.4.2     ✔ tibble    3.2.1
## ✔ lubridate 1.9.2     ✔ tidyr     1.3.0
## ✔ purrr     1.0.1     
## ── Conflicts ───────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package to force all conflicts to become errors

That’s all helpful information—it tells you that R loaded 9 related packages for you ({ggplot2}, {dplyr}, etc.). But none of that needs to be in a knitted document. You can turn off those messages and warnings using chunk options:

```{r load-packages, warning=FALSE, message=FALSE}
library(tidyverse)
```

The same technique works for other messages too. In exercise 3, for instance, you saw this message a lot:

## `summarise()` has grouped output by 'Gender'. 
## You can override using the .groups` argument.

That’s nothing bad and you did nothing wrong—that’s just R talking to you and telling you that it did something behind the scenes. When you use group_by() with one variable, like group_by(Gender), once you’re done summarizing and working with the groups, R ungroups your data automatically. When you use group_by() with two variables, like group_by(Gender, Film), once you’re done summarizing and working with the groups, R ungroups the last of the variables and gives you a data frame that is still grouped by the other variables. So with group_by(Gender, Film), after you’ve summarized stuff, R stops grouping by Film and groups by just Gender. That’s all the summarise() has grouped output by... message is doing—it’s telling you that it’s still grouped by something. It’s no big deal.

So, to get rid of the message in this case, you can use message=FALSE in the chunk options to disable the message:

```{r lotr-use-two-groups, message=FALSE}
lotr_gender_film <- lotr %>% 
  group_by(Gender, Film) %>% 
  summarize(total = sum(Words))
```

Why did we need to group a second time when calculating the proportions for the heatmap?

At the end of exercise 4, you created a heatmap showing the proportions of different types of construction projects across different boroughs. In the instructions, I said you’d need to use group_by() twice to get predictable proportions. Some of you have wondered what this means. Here’s a quick illustration.

When you group by a column, R splits your data into separate datasets behind the scenes, and when you use summarize(), it calculates summary statistics (averages, counts, medians, etc.) for each of those groups. So when you used group_by(BOROUGH, CATEGORY), R made smaller datasets of Bronx Affordable Housing, Bronx Approved Work, Brooklyn Affordable Housing, Brooklyn Approved Work, and so on. Then when you used summarize(total = n()), you calculated the number of rows in each of those groups, thus giving you the number of projects per borough per category. That’s basic group_by() %>% summarize() stuff.

Once you have a count of projects per borough, you have to decide how you want to calculate proportions. In particular, you need to figure out what your denominator is. Do you want the proportion of all projects within each borough (e.g. X% of projects in the Bronx are affordable housing, Y% in the Bronx are approved work, and so on until 100% of the projects in the Bronx are accounted for), so that each borough constitutes 100%? Do you want the proportion of boroughs for each project (e.g. X% of affordable housing projects are in the Bronx, Y% of affordable housing projects are in Brooklyn, and so on until 100% of the affordable housing projects are accounted for). This is where the second group_by() matters.

For example, if you group by borough and then use mutate to calculate the proportion, the proportion in each borough will add up to 100%. Notice the denominator column here—it’s unique to each borough (1169 for the Bronx, 2231 for Brooklyn, etc.).

essential %>%
  group_by(BOROUGH, CATEGORY) %>%
  summarize(totalprojects = n()) %>%
  group_by(BOROUGH) %>%
  mutate(denominator = sum(totalprojects),
         proportion = totalprojects / denominator)
#> # A tibble: 33 × 5
#> # Groups:   BOROUGH [5]
#>    BOROUGH  CATEGORY               totalprojects denominator proportion
#>    <fct>    <fct>                          <int>       <int>      <dbl>
#>  1 Bronx    Affordable Housing                80        1169   0.0684  
#>  2 Bronx    Approved Work                    518        1169   0.443   
#>  3 Bronx    Homeless Shelter                   1        1169   0.000855
#>  4 Bronx    Hospital / Health Care            55        1169   0.0470  
#>  5 Bronx    Public Housing                   276        1169   0.236   
#>  6 Bronx    Schools                          229        1169   0.196   
#>  7 Bronx    Utility                           10        1169   0.00855 
#>  8 Brooklyn Affordable Housing               168        2231   0.0753  
#>  9 Brooklyn Approved Work                   1223        2231   0.548   
#> 10 Brooklyn Hospital / Health Care            66        2231   0.0296  
#> # … with 23 more rows

If you group by category instead, the proportion within each category will add to 100%. Notice how the denominator for affordable housing is 372, approved work is 4189, and so on.

essential %>%
  group_by(BOROUGH, CATEGORY) %>%
  summarize(totalprojects = n()) %>%
  group_by(CATEGORY) %>%
  mutate(denominator = sum(totalprojects),
         proportion = totalprojects / denominator)
#> # A tibble: 33 × 5
#> # Groups:   CATEGORY [7]
#>    BOROUGH  CATEGORY               totalprojects denominator proportion
#>    <fct>    <fct>                          <int>       <int>      <dbl>
#>  1 Bronx    Affordable Housing                80         372      0.215
#>  2 Bronx    Approved Work                    518        4189      0.124
#>  3 Bronx    Homeless Shelter                   1           5      0.2  
#>  4 Bronx    Hospital / Health Care            55         259      0.212
#>  5 Bronx    Public Housing                   276        1014      0.272
#>  6 Bronx    Schools                          229        1280      0.179
#>  7 Bronx    Utility                           10          90      0.111
#>  8 Brooklyn Affordable Housing               168         372      0.452
#>  9 Brooklyn Approved Work                   1223        4189      0.292
#> 10 Brooklyn Hospital / Health Care            66         259      0.255
#> # … with 23 more rows

You can also ungroup completely before calculating the proportion. This makes it so the entire proportion column adds to 100%:

essential %>%
  group_by(BOROUGH, CATEGORY) %>%
  summarize(totalprojects = n()) %>%
  ungroup() %>%
  mutate(denominator = sum(totalprojects),
         proportion = totalprojects / denominator)
#> # A tibble: 33 × 5
#>    BOROUGH  CATEGORY               totalprojects denominator proportion
#>    <fct>    <fct>                          <int>       <int>      <dbl>
#>  1 Bronx    Affordable Housing                80        7209   0.0111  
#>  2 Bronx    Approved Work                    518        7209   0.0719  
#>  3 Bronx    Homeless Shelter                   1        7209   0.000139
#>  4 Bronx    Hospital / Health Care            55        7209   0.00763 
#>  5 Bronx    Public Housing                   276        7209   0.0383  
#>  6 Bronx    Schools                          229        7209   0.0318  
#>  7 Bronx    Utility                           10        7209   0.00139 
#>  8 Brooklyn Affordable Housing               168        7209   0.0233  
#>  9 Brooklyn Approved Work                   1223        7209   0.170   
#> 10 Brooklyn Hospital / Health Care            66        7209   0.00916 
#> # … with 23 more rows

Which one you do is up to you—it depends on the story you’re trying to tell.

Slack in public

One request regarding Slack: lots of you have been sending me private messages there for assistance, and that’s great and I’m happy to respond like that. However, one way to help build a stronger community for this class is to ask questions in public in the #help channel instead. There are a couple reasons for this:

1. It reduces duplication: Many of you have had almost identical questions and I’ve been able to copy/paste screenshots between different private conversations asking for help. Having those questions and answers in #help instead will let you get answers to common questions faster
2. It allows you to help: Some of you have R experience already, and even if you don’t, as the summer goes on, you’ll get more comfortable with it and will start being able to answer your classmates’ questions. You might have just fixed a similar issue in a past exercise, or you might be able to spot a typo in their code, or you might otherwise know how to help. Step in and help! Slack is for building a community, not just for getting assistance from me.

Knitted document format

R Markdown is great and wonderful in part because you can write a document in one .Rmd file and have it magically turn into an HTML file, a Word file, or a PDF (or if you want to get extra fancy later, a slideshow, a dashboard, or even a full website).

iCollege doesn’t like HTML files, though. It won’t show images that get uploaded because of weird server restrictions or something. So when you submit your exercises, make sure you knit to PDF or Word:

I’d recommend knitting to HTML often as you work on the exercise. Knitting to PDF takes a few extra seconds, and knitting to Word is a hassle because Word gets mad if you have a previous version of the document open when knitting. HTML is pretty instantaneous. When I work on R Markdown files, I put a browser window on one of my monitors and RStudio on the other and knit and reknit often to HTML while working. Once I’m done and the document all works and the images, tables, text, etc. are all working, I’ll knit to PDF or Word or whatever final format I want.

Column names and the hyperliterality of computers

Computers are incredibly literal and they cannot read your mind! As humans we’re good at figuring out information when data is missing or distorted—if someone sends you a text saying that they’re “running 5 minutes latte”, you know that they’re not running with some weird time-based coffee but are instead running late. Computers can’t figure that out.

For example, in Exercise 1 if you want to make a plot showing the relationship between highway MPG and cylinders, if you type anything like this, it won’t work.

Here there’s an error because there are no columns named cylinders or highway:

ggplot(data = cars, mapping = aes(x = cylinders, y = highway)) +
  geom_point()
## Error in `geom_point()`:
## ! Problem while computing aesthetics.
## ℹ Error occurred in the 1st layer.
## Caused by error:
## ! object 'cylinders' not found

Here there’s an error because there is no column named Cyl. It’s cyl with a lowercase c. R is case sensitive.

ggplot(data = cars, mapping = aes(x = Cyl, y = hwy)) +
  geom_point()
## Error in `geom_point()`:
## ! Problem while computing aesthetics.
## ℹ Error occurred in the 1st layer.
## Caused by error:
## ! object 'Cyl' not found

The easiest way to make sure you’re using the right column names is to look at them. RStudio gives you a couple easy ways to do this. After you’ve loaded your data, you should see it in your environment panel:

If you click on the little blue arrow next to the name of the dataset, RStudio will show you the column names and the first few values in each:

If you click on the name of the dataset (cars here), RStudio will open a new tab with a read-only view of the data. You can click on the column names to sort and filter things too, which is convenient.

I constantly have to refer back to the list of column names to make sure I’m spelling things correctly and using the right capitalization.

The magic of labs()

A few years ago, if you wanted to add labels to a plot with ggplot, you had to use specific functions, like xlab() and ylab() for the axis labels and ggtitle() for the plot title, like this:

library(tidyverse)

# The old way; don't do this!
ggplot(mpg, aes(x = cyl, y = displ, color = drv)) +
  geom_point() +
  xlab("Cylinders") +
  ylab("Displacement") +
  ggtitle("A plot showing stuff")

Lots of older code examples on the internet will still use these functions. However, those have all been replaced with a newer, nicer labs() function that lets you control all the different plot labels in one layer. Here’s how to make that same plot above with labs() instead:

# The better way; do this!
ggplot(mpg, aes(x = cyl, y = displ, color = drv)) +
  geom_point() +
  labs(
    x = "Cylinders",
    y = "Displacement",
    title = "A plot showing stuff"
  )

The labs() function also lets you do subtitles and captions and lets you control the legend titles too. Here we mapped drv to the color aesthetic, so we can change its title by using color in labs():

# The better way; do this!
ggplot(mpg, aes(x = cyl, y = displ, color = drv)) +
  geom_point() +
  labs(
    x = "Cylinders",
    y = "Displacement",
    title = "A plot showing stuff",
    subtitle = "Super neat",
    caption = "I still have no idea what displacement is",
    color = "Drive"
  )

Will we cover {X}?

• Geospatial data and visualizations? Absolutely! Maps are one of my favorite kinds of visualization and we have a whole session on them.
• Interactive visualizations? Yep!
• Animated visualizations? Not directly, but we’ll briefly mention the {gganimate} package, which is surprisingly easy to use—it’s just regular ggplot code with a few extra lines to animate it

Will you have regular office hours?

I don’t have a set time every week for student hours (see here for why I don’t call them office hours). Instead, I use an appointment system. If you want to meet with me online, visit my Calendly page and sign up for a slot. There’s also a link to it on the homepage and syllabus page of the class website.

I’ll also occasionally be on campus starting the last week of June (I’m currently not in Atlanta—I’m in Utah visiting family, but will still be responding to all e-mails and providing support and answering questions via e-mail and Slack), so if you want to meet in person after June 26, we can arrange that too.

I don’t have a background in anything related to PMAP (public management and policy)—will I be lost?

That’s 1000% fine! I’ve designed this class to be accessible and useful for people working with any kind of data. Lots of the examples we’ll play with will often be related to political science-y things, but not always. And the kind of data doesn’t actually matter. When learning new statistical programming and visualization tools and techniques, the industry standard is to work with toy datasets. For instance, in your first exercise you’ll make a plot of car mileage. I know nothing about cars at all, but I (along with everyone else in the R world) use that dataset all the time for quick little tests of plots or models or code or whatever, since it’s built into R. I also often use data about penguins and lots of other domains in assignments and examples.

Don’t worry about the precise nature of the data. In your final project, you’ll make visualizations with whatever data you want. For all the other assignments and exercises and mini-projects, just go with the data I’ve provided. It’s all actual real world data (refugee inflows, unemployment, election results, texts of books from the 19th century, counts and locations of rat sightings in NYC, etc.). If it’s not related to your field (hello you handful of epidemiology students!), don’t worry—the principles of code and data visualization and graphic design still apply.

Can I reach out if I have questions?

ABSOLUTELY. DO NOT ATTEMPT TO DO THIS CLASS WITHOUT EVER ASKING A SINGLE QUESTION.

Throughout the pandemic year I’ve been studying the research on online teaching to see how to make my online classes as successful as possible. One almost universal key to success I’ve found is that learning requires social interaction to be successful (Glazier 2016). In fact, one of the greatest predictors of success is whether students build relationships with their classmates and their professor.

This is true for both online classes and face-to-face classes! It’s entirely possible to take a face-to-face class and have almost no social interaction. The worst, most useless class I ever took as an undergraduate was a required Intro to US History class that met twice a week for 2 hours in a massive auditorium with 900 (!!!) other students. It was awful and I learned nothing.

So, interact with me and your classmates! Don’t lurk in silence in the shadows. (Don’t overshare either and spam the #general channel in Slack, but in my experience there’s little risk of that). Do reach out and talk to your classmates (and me!). We’re an eclectic group. Even though this is officially an Andrew Young School class, most of you are not public policy or public administration grad students. We have PhD students in political science and epidemiology and masters students in criminal justice administration, public policy, and economics. Get to know each other!

To that end, one of the best ways to get to know each other is to help each other. Ask questions on Slack. If you see someone’s question there and you know how to help, answer!

You can also work on exercises and projects in groups if you want. Use Webex or Zoom to share screens with each other during calls so you can see what you’re doing. In the real programming world, a common technique for working on code is pair programming, where two people work on the same script. If we were meeting in class, we’d do this face-to-face, working during class in groups of 2–3 with me walking around and helping. Feel free to replicate that experience and work with others. It’s not cheating! (As long as you don’t turn in identical work)

Help make this semester fantastic by talking, reaching out, and otherwise socially interacting!