13 U.S. Census Bureau

In this chapter, we’ll use the tidycensus package. You’ll need to install it if you haven’t already.

13.1 Census Bureau basics

The U.S. Census Bureau is a fantastic resource for data related to the U.S. population. As you saw in the API Basics chapter, the Census Bureau makes a wide variety of APIs available. In this chapter, we’ll focus on three: the decennial census, the American Community Survey, and the population estimates.

As we mentioned previously, many R packages wrap commonly used APIs, making it easier for you to obtain data. In this chapter, we’ll introduce one such package: tidycensus. We’ll show you how to use tidycensus to obtain data from the decennial census and American Community Survey. Then, we’ll go into more detail about working directly with the Census Bureau APIs.

First, we’ll give a bit of background about three U.S. Census data sources: the decennial census, the American Community Survey, and the population estimates.

13.1.1 Decennial census

When most people think of the U.S. Census, they’re thinking about the decennial census. The Census Bureau conducts the decennial census every ten years (starting in 1790), with the goal of determining the number of people living in the United States. Because many features of the U.S. government, including the number of representatives awarded to each state, depend on accurate population counts, the decennial census is required by the Constitution.

For the decennial census, the Census Bureau tries to survey every household in the U.S. in an attempt to count every U.S. resident. The population estimates that come from the decennial census are therefore the most definitive that you can find. However, decennial census data come out only every ten years, so can be out-of-date. The decennial census survey also only asks a few questions, primarily about household size, race, ethnicity.1 The American Community Survey provides more detailed and up-to-date data.

13.1.2 ACS

Between 1790 and 2000, decennial censuses included both a short form and a long form. Every household filled out the short form, but a sample also filled out the long form, which included additional questions. After 2000, the Census Bureau turned the long form into the American Community Survey (ACS), and began administering the ACS every year.

The ACS, unlike the decennial census, is a sample. Every year, the ACS surveys a representative sample consisting of 3.5 million households. The Census Bureau then uses this sample to provide estimates for the entire U.S. population.2

The ACS calculates these estimates over two time periods: 1 year and 5 years. The 1-year estimates are the most current, but have larger margins of error due to their smaller sample size. Most of the time, you’ll use the 5-year estimates. Their larger sample sizes gives them greater accuracy, particularly for smaller geographic units.

13.1.3 Population estimates

The third data source we’ll discuss comes from the Census Bureau’s Population Estimates Program (PEP). The decennial census publishes the definitive population of the United States every ten years. However, if you want to know the population of a U.S. geographic area between decennial census years, you’ll need to use the Population Estimates APIs.

The ACS also includes population estimates, but the estimation techniques used for the Population Estimates Program are more accurate.

13.1.4 Choosing data

The first step to working with U.S. Census data is to decide which data source to use. Here’s a quick guide:

ACS

Most of the time, you’ll want the ACS. The ACS includes many different variables on social, economic, housing, and demographic aspects.

You’ll typically want the 5-year ACS, unless you’re looking at a large or rapidly changing geographic area or need yearly data.

Decennial census

Use the decennial census if you want definitive population data, don’t need that many variables, and don’t mind that the data is only available every 10 years.

Population estimates

Use data from the Population Estimates Program if you want accurate population data for a non-decennial census year (i.e., a year not divisible by 10).

13.2 tidycensus

The tidycensus package wraps several U.S. Census Bureau APIs, allowing you to access decennial census and ACS data through R functions.

Before you use tidycensus for the first time, you’ll need to obtain a Census Bureau API key. You can request one here. You’ll receive an email with your key. Copy your key to the clipboard, then navigate back to RStudio. Run the following line to open your .Renviron:

Then, add the following line, replacing YOUR_API_KEY with the key sent to you by the Census Bureau.

Restart R (Ctrl/Cmd + Shift + F10) for the changes to take effect.

From now on, you won’t need to worry about a key. Your key will stay in your .Renviron across R sessions.

13.2.1 Specify a dataset

We’ll show you how to use the tidycensus package to access two Census APIs: the decennial census and the ACS. Before you start using tidycensus, you’ll need to decide which dataset and year to use.

Dataset

See our discussion in Choosing data for the trade-off between the decennial census, ACS, and the different ACS estimates.

For many tidycensus functions, you specify the different surveys in the following way:

  • "acs5": 5-year ACS
  • "acs1": 1-year ACS
  • "sf1": Decennial census

sf stands for Summary File. Summary File 1 ("sf1") corresponds to the short form described earlier, while Summary File 3 ("sf3") corresponds to the long form. As explained earlier, the ACS took the place of the long form in 2001, so "sf3" is only available for censuses from 2000 or earlier.

Year

You’ll also need to decide on a year. For the decennial censuses, year will just be the year of the decennial census. Remember that the decennial census occurs in years ending in 0. The tidycensus package can access the 1990, 2000, and 2010 decennial censuses.

For ACS data, the year argument of tidycensus functions refers to the end-year of the sample period. For example, if you want to use a 5-year ACS that ended in 2017, set year = 2017. As of January 2020, tidycensus supports the end-years 2010 through 2017.

13.2.2 Find variables

Data from both the ACS and decennial census capture many variables. The 2010 decennial census includes 8,960, while the 2017 5-year ACS includes 25,070!

A code, like H001001 or P011014, identifies each of these variables. To use tidycensus, you’ll need to determine the codes of your variables of interest. We’ll use the function tidycensus::load_variables() to find ACS or decennial census variables and their accompanying codes.

load_variables() returns a tibble of all variable codes from a given dataset, alongside brief descriptions. We’ll load the variables from the 2017 5-year ACS as an example.

#> # A tibble: 25,070 x 3
#>   name       label                           concept                            
#>   <chr>      <chr>                           <chr>                              
#> 1 B00001_001 Estimate!!Total                 UNWEIGHTED SAMPLE COUNT OF THE POP…
#> 2 B00002_001 Estimate!!Total                 UNWEIGHTED SAMPLE HOUSING UNITS    
#> 3 B01001_001 Estimate!!Total                 SEX BY AGE                         
#> 4 B01001_002 Estimate!!Total!!Male           SEX BY AGE                         
#> 5 B01001_003 Estimate!!Total!!Male!!Under 5… SEX BY AGE                         
#> 6 B01001_004 Estimate!!Total!!Male!!5 to 9 … SEX BY AGE                         
#> # … with 2.506e+04 more rows

load_variables() returns a tibble with the three variables:

  • name: The variable code.
  • label: A description of the variable.
  • concept: A broader categorization.

Let’s take a closer look at just one concept: sex by age.

#> # A tibble: 49 x 3
#>   name       label                                 concept   
#>   <chr>      <chr>                                 <chr>     
#> 1 B01001_001 Estimate!!Total                       SEX BY AGE
#> 2 B01001_002 Estimate!!Total!!Male                 SEX BY AGE
#> 3 B01001_003 Estimate!!Total!!Male!!Under 5 years  SEX BY AGE
#> 4 B01001_004 Estimate!!Total!!Male!!5 to 9 years   SEX BY AGE
#> 5 B01001_005 Estimate!!Total!!Male!!10 to 14 years SEX BY AGE
#> 6 B01001_006 Estimate!!Total!!Male!!15 to 17 years SEX BY AGE
#> # … with 43 more rows

49 variables belong to the “SEX BY AGE” concept. Each row refers to a variable under that concept.

This way of thinking about variables can be a bit confusing at first. The Estimate!!Total variable captures the number of people for whom sex by age data is available. Estimate!!Total!!Male captures the total number of males, while Estimate!!Total!!Male!!Under 5 years captures the total number of males under 5 years old.

This is a bit more intuitive for concepts like "SEX BY AGE (ASIAN ALONE)".

#> # A tibble: 31 x 3
#>   name        label                                 concept                 
#>   <chr>       <chr>                                 <chr>                   
#> 1 B01001D_001 Estimate!!Total                       SEX BY AGE (ASIAN ALONE)
#> 2 B01001D_002 Estimate!!Total!!Male                 SEX BY AGE (ASIAN ALONE)
#> 3 B01001D_003 Estimate!!Total!!Male!!Under 5 years  SEX BY AGE (ASIAN ALONE)
#> 4 B01001D_004 Estimate!!Total!!Male!!5 to 9 years   SEX BY AGE (ASIAN ALONE)
#> 5 B01001D_005 Estimate!!Total!!Male!!10 to 14 years SEX BY AGE (ASIAN ALONE)
#> 6 B01001D_006 Estimate!!Total!!Male!!15 to 17 years SEX BY AGE (ASIAN ALONE)
#> # … with 25 more rows

Here, Estimate!!Total represents the total number of Asian U.S. residents for whom sex/age data is relevant.

Most concepts have a Estimate!!Total variable, or something similar. If you want to calculate a proportion, such as the proportion of males, use the relevant Estimate!!Total as the denominator.

To find the variables you want, pipe the result of load_variables() into view().

You can use the search bar to search for variables with a given a keyword (e.g., “income”). You can also click on the Filter button to get a search bar for each variable. Once you’ve found the variables you want, copy their codes (the name variable), and store them in a named vector.

If we wanted to get variables from, say the 2010 decennial census, we’d use

The documentation for the ACS is helpful if you need additional information about ACS variables.

13.2.3 Get data

tidycensus includes two functions for obtaining data from the Census Bureau: get_acs() and get_decennial().

At minimum, you should supply get_acs() and get_decennial() with three variables:

  • geography
  • variables
  • year

geography controls the geographic level of the data returned. The tidycensus website includes a helpful table of all available geographics. Common values are “state” and “county”.

Supply variables with a vector of variable codes. Earlier, we stored the codes of some variables in the vector vars_acs5.

#> median_income   median_rent 
#>  "B06011_001"  "B25064_001"

For get_acs(), year indicates the end-year for the ACS estimates. If you want ACS estimates from 2012-2017, set year = 2017. By default, get_acs() uses the 5-year estimates. You can use other estimates by specifying survey.

#> Getting data from the 2013-2017 5-year ACS
#> # A tibble: 104 x 5
#>   GEOID NAME    variable      estimate   moe
#>   <chr> <chr>   <chr>            <dbl> <dbl>
#> 1 01    Alabama median_income    24476   136
#> 2 01    Alabama median_rent        747     3
#> 3 02    Alaska  median_income    32940   508
#> 4 02    Alaska  median_rent       1200    13
#> 5 04    Arizona median_income    27517   148
#> 6 04    Arizona median_rent        972     4
#> # … with 98 more rows

get_acs() will return the estimate and margin of error (moe) for each variable. Because the ACS values are estimates, the Census Bureau calculates a margin of error for most variables.

To pivot the data into a wider format, we can use pivot_wider().

#> # A tibble: 52 x 6
#>   GEOID NAME  estimate_median… estimate_median… moe_median_inco… moe_median_rent
#>   <chr> <chr>            <dbl>            <dbl>            <dbl>           <dbl>
#> 1 01    Alab…            24476              747              136               3
#> 2 02    Alas…            32940             1200              508              13
#> 3 04    Ariz…            27517              972              148               4
#> 4 05    Arka…            23789              709              165               5
#> 5 06    Cali…            29454             1358              109               3
#> 6 08    Colo…            32401             1125              109               5
#> # … with 46 more rows

get_decennial() works similarly. First, we’ll find the variables with load_variables() and store several in a vector.

Then, we’ll use get_decennial() to access the data. This time, we’ll get the data at the county level. Here, year is the year of the decennial census.

#> Getting data from the 2010 decennial Census
#> # A tibble: 6,442 x 4
#>   GEOID NAME                       variable  value
#>   <chr> <chr>                      <chr>     <dbl>
#> 1 05131 Sebastian County, Arkansas pop_urban 43495
#> 2 05133 Sevier County, Arkansas    pop_urban  2115
#> 3 05135 Sharp County, Arkansas     pop_urban  2177
#> 4 05137 Stone County, Arkansas     pop_urban     0
#> 5 05139 Union County, Arkansas     pop_urban  8830
#> 6 05141 Van Buren County, Arkansas pop_urban     0
#> # … with 6,436 more rows

Again, we can use pivot_wider().

#> # A tibble: 3,221 x 4
#>   GEOID NAME                       pop_urban pop_rural
#>   <chr> <chr>                          <dbl>     <dbl>
#> 1 05131 Sebastian County, Arkansas     43495     11156
#> 2 05133 Sevier County, Arkansas         2115      4772
#> 3 05135 Sharp County, Arkansas          2177      7645
#> 4 05137 Stone County, Arkansas             0      6712
#> 5 05139 Union County, Arkansas          8830     10823
#> 6 05141 Van Buren County, Arkansas         0     10345
#> # … with 3,215 more rows

Now, you can use your data however you wish. Note that, if you’re interested in geospatial aspects of ACS or decennial census data, we recommend using our ussf package for boundaries. You can install the package with the following command.

You can join your ACS or decennial census data with the result of ussf::boundaries(), using GEOID as a unique identifier.

#> # A tibble: 104 x 6
#>   GEOID NAME   variable  estimate   moe                                 geometry
#>   <chr> <chr>  <chr>        <dbl> <dbl>                       <MULTIPOLYGON [m]>
#> 1 01    Alaba… median_i…    24476   136 (((708460 -598743, 708114 -594376, 7084…
#> 2 01    Alaba… median_r…      747     3 (((708460 -598743, 708114 -594376, 7084…
#> 3 02    Alaska median_i…    32940   508 (((-1518019 -1148523, -1516049 -1146136…
#> 4 02    Alaska median_r…     1200    13 (((-1518019 -1148523, -1516049 -1146136…
#> 5 04    Arizo… median_i…    27517   148 (((-1743454 -375819, -1743828 -373071, …
#> 6 04    Arizo… median_r…      972     4 (((-1743454 -375819, -1743828 -373071, …
#> # … with 98 more rows

13.3 Population estimates

For population estimates outside decennial census years, you’ll need to work directly with the Population Estimates APIs.

In the API Basics chapter, we discussed a workflow for working directly with APIs, and walked you through an example that used the Population Estimates. In this section, we’ll dive into more detail about the Population Estimates APIs.

13.3.1 Choose data

There are two types of population estimate APIs: the vintages and the intercensals. Each vintage contains data on all years since the last decennial census. For example, the 2018 Vintage contains data for each year between 2010 and 2018. You should use the most recent vintage available, since the Census Bureau updates all previous years’ estimates.

The intercensals contain data between previous decennial census. For example, the 2000-2010 intercensal contains yearly data for 2000 through 2010. If you want data from, for example, 2000 to 2018, you’ll need to use both a vintage and an intercensal, which will involve two API queries.

As of February 2020, the Vintage 2018 estimates are the most recent, fully available estimates. Vintage 2019 estimates are available for some geographic units, but won’t be available at the county level until March 2020.

Once you’ve decided on a vintage or a intercensal, click on the corresponding tab. You’ll see the various available APIs for those estimates.

The links under the API name provide you with more information about the API. If you want population data, you’ll probably want the Population Estimates API, which provides yearly population estimates at various geographic levels.

13.3.2 Craft your request

In the Census Basics chapter, we laid out the steps for working with an API. After finding your API, the next step is to craft the request.

First, you’ll need to find the base request, which will be listed next to API Call.

Next, add parameters to the base request to specify exactly what data you want. For the Population Estimates APIs, there are two important parameters: get and for. (You’ll also see key in the examples on the Census website, but an API key isn’t actually necessary.)

get controls which variables (i.e., what will eventually become your tibble columns) the request returns. for controls the geographic level. The parameters come after a ? and are separated by a &. We’ll go over both in more detail next.

Variables

To see all possible variables, click on the Variables link under your chosen API.

This will lead you to a table of variables.

The variables in all caps, like POP and GEONAME, are the names of variables returned by the API. The variables in lowercase, like for and in, are actually API request parameters. For now, just pay attention to the uppercase variables. You’ll specify the names of your desired variable after the API parameter get.

Often, you’ll want to get population over time. To get data for each year, you’ll need the variables DATE_DESC and DATE_CODE. DATE_CODE is a code associated with a date (e.g., 1 or 2), and DATE_DESC describes the date (e.g., "7/1/2018 population estimate"). Without DATE_DESC, you won’t know what each DATE_CODE refers to, and without DATE_CODE, the API only returns data for one year.

Geographies

Next, you’ll need to specify what geographic level of the data. For each Population Estimate API, check which geographies are available by clicking on the Examples and Supported Geographies link.

This link leads to a table with a description of the API, and links to examples, geographies, and other information. Click on the geographies link. Then, you’ll see a table with different geographies.

Use these geographies to determine the data returned. For example:

  • for=us will return data for the entire U.S.
  • for=state:30 will return data for Montana.
  • for=state:* will return data by state, for all states.
  • for=county:* will return data by county, for all counties in the U.S.
  • for=county:*&in=state:30 will return data by county, just for Montana.

13.3.2.1 Examples

Each API has an examples page. From the Population Estimates homepage, navigate to Examples and Supported Geographies > Examples. This page lists examples for the 2018 Vintage Population Estimates API.

Here are some more examples:

Entire U.S. population data by year

https://api.census.gov/data/2018/pep/population?get=GEONAME,DATE_CODE,DATE_DESC,POP&for=us

Population data for only Montana, by year

https://api.census.gov/data/2018/pep/population?get=GEONAME,DATE_CODE,DATE_DESC,POP&for=state:30

Population data for all counties, by year

https://api.census.gov/data/2018/pep/population?get=GEONAME,DATE_CODE,DATE_DESC,POP&for=county:*

13.3.3 Read the data into R

The next step is to read your data into R. We’ll use the same process introduced in API Basics. Let’s use a request that gets population data for all states, by year.

#> # A tibble: 572 x 5
#>   GEONAME DATE_CODE DATE_DESC                          POP     state
#>   <chr>   <chr>     <chr>                              <chr>   <chr>
#> 1 Alabama 1         4/1/2010 Census population         4779736 01   
#> 2 Alabama 2         4/1/2010 population estimates base 4780138 01   
#> 3 Alabama 3         7/1/2010 population estimate       4785448 01   
#> 4 Alabama 4         7/1/2011 population estimate       4798834 01   
#> 5 Alabama 5         7/1/2012 population estimate       4815564 01   
#> 6 Alabama 6         7/1/2013 population estimate       4830460 01   
#> # … with 566 more rows

You’ll still need to do a bit of light cleaning. Also, notice that there are three different estimates for 2010 (the decennial census year).

#>  [1] "4/1/2010 Census population"         "4/1/2010 population estimates base"
#>  [3] "7/1/2010 population estimate"       "7/1/2011 population estimate"      
#>  [5] "7/1/2012 population estimate"       "7/1/2013 population estimate"      
#>  [7] "7/1/2014 population estimate"       "7/1/2015 population estimate"      
#>  [9] "7/1/2016 population estimate"       "7/1/2017 population estimate"      
#> [11] "7/1/2018 population estimate"

You’ll generally want to use all the July 1st estimates so that all estimates are a year apart.