Pew Research Center (PEW)

Build Status Build status

The Pew Research Center releases its survey microdata on U.S. Politics & Policy, Journalism & Media, Internet, Science & Tech, Religion & Public Life, Hispanic Trends, Global Attitudes & Trends, and Social & Demographic Trends.

  • Generally one table per survey, with one row per sampled respondent.

  • Complex sample surveys, often designed to generalize to the U.S. adult population or the adult populations of the nations surveyed.

  • Administered by the Pew Research Center.

Simplified Download and Importation

The R lodown package easily downloads and imports all available PEW microdata by simply specifying "pew" with an output_dir = parameter in the lodown() function. Depending on your internet connection and computer processing speed, you might prefer to run this step overnight.

library(lodown)
lodown( "pew" , output_dir = file.path( path.expand( "~" ) , "PEW" ) )

lodown also provides a catalog of available microdata extracts with the get_catalog() function. After requesting the PEW catalog, you could pass a subsetted catalog through the lodown() function in order to download and import specific extracts (rather than all available extracts).

library(lodown)
# examine all available PEW microdata files
pew_cat <-
    get_catalog( "pew" ,
        output_dir = file.path( path.expand( "~" ) , "PEW" ) )

# spring 2015 only
pew_cat <- subset( pew_cat , name == "Spring 2015 Survey Data" )
# download the microdata to your local computer
pew_cat <- lodown( "pew" , pew_cat )

Analysis Examples with the survey library  

Construct a complex sample survey design:

options( survey.lonely.psu = "adjust" )

library(survey)

pew_df <- 
    readRDS( 
        file.path( path.expand( "~" ) , "PEW" , 
        "Global Attitudes & Trends/2015/Spring 2015 Survey Data" ,
        "Pew Research Global Attitudes Spring 2015 Dataset for Web FINAL.rds" )
    )

# limit the global attitudes data set to just israel
israel_df <- subset( pew_df , country == 14 )
    
pew_design <- 
    svydesign(
        id = ~ psu , 
        strata = ~ stratum , 
        weight = ~ weight , 
        data = israel_df 
    )

Variable Recoding

Add new columns to the data set:

pew_design <- 
    update( 
        pew_design , 
        
        one = 1 ,
        
        your_day_today =
            factor( 
                q1 , 
                levels = 1:3 ,
                labels = 
                    c( 
                        'a typical day' , 
                        'a particularly good day' , 
                        'a particularly bad day' 
                    )
            ) ,

        school_years = ifelse( q163b %in% 98:99 , NA , q163b ) ,
        
        age_in_years = ifelse( q146 %in% 98:99 , NA , q146 ) ,

        climate_change_concern = ifelse( q13a %in% 1:5 , as.numeric( q13a < 3 ) , NA ) ,
        
        country_economic_situation =
            factor(
                q3 ,
                levels = 1:4 ,
                labels = 
                    c( 
                        'very good' , 
                        'somewhat good' , 
                        'somewhat bad' , 
                        'very bad' 
                    )
            )
    )

Unweighted Counts

Count the unweighted number of records in the survey sample, overall and by groups:

sum( weights( pew_design , "sampling" ) != 0 )

svyby( ~ one , ~ your_day_today , pew_design , unwtd.count )

Weighted Counts

Count the weighted size of the generalizable population, overall and by groups:

svytotal( ~ one , pew_design )

svyby( ~ one , ~ your_day_today , pew_design , svytotal )

Descriptive Statistics

Calculate the mean (average) of a linear variable, overall and by groups:

svymean( ~ school_years , pew_design , na.rm = TRUE )

svyby( ~ school_years , ~ your_day_today , pew_design , svymean , na.rm = TRUE )

Calculate the distribution of a categorical variable, overall and by groups:

svymean( ~ country_economic_situation , pew_design , na.rm = TRUE )

svyby( ~ country_economic_situation , ~ your_day_today , pew_design , svymean , na.rm = TRUE )

Calculate the sum of a linear variable, overall and by groups:

svytotal( ~ school_years , pew_design , na.rm = TRUE )

svyby( ~ school_years , ~ your_day_today , pew_design , svytotal , na.rm = TRUE )

Calculate the weighted sum of a categorical variable, overall and by groups:

svytotal( ~ country_economic_situation , pew_design , na.rm = TRUE )

svyby( ~ country_economic_situation , ~ your_day_today , pew_design , svytotal , na.rm = TRUE )

Calculate the median (50th percentile) of a linear variable, overall and by groups:

svyquantile( ~ school_years , pew_design , 0.5 , na.rm = TRUE )

svyby( 
    ~ school_years , 
    ~ your_day_today , 
    pew_design , 
    svyquantile , 
    0.5 ,
    ci = TRUE ,
    keep.var = TRUE ,
    na.rm = TRUE
)

Estimate a ratio:

svyratio( 
    numerator = ~ school_years , 
    denominator = ~ age_in_years , 
    pew_design ,
    na.rm = TRUE
)

Subsetting

Restrict the survey design to seniors:

sub_pew_design <- subset( pew_design , q146 >= 65 )

Calculate the mean (average) of this subset:

svymean( ~ school_years , sub_pew_design , na.rm = TRUE )

Measures of Uncertainty

Extract the coefficient, standard error, confidence interval, and coefficient of variation from any descriptive statistics function result, overall and by groups:

this_result <- svymean( ~ school_years , pew_design , na.rm = TRUE )

coef( this_result )
SE( this_result )
confint( this_result )
cv( this_result )

grouped_result <-
    svyby( 
        ~ school_years , 
        ~ your_day_today , 
        pew_design , 
        svymean ,
        na.rm = TRUE 
    )
    
coef( grouped_result )
SE( grouped_result )
confint( grouped_result )
cv( grouped_result )

Calculate the degrees of freedom of any survey design object:

degf( pew_design )

Calculate the complex sample survey-adjusted variance of any statistic:

svyvar( ~ school_years , pew_design , na.rm = TRUE )

Include the complex sample design effect in the result for a specific statistic:

# SRS without replacement
svymean( ~ school_years , pew_design , na.rm = TRUE , deff = TRUE )

# SRS with replacement
svymean( ~ school_years , pew_design , na.rm = TRUE , deff = "replace" )

Compute confidence intervals for proportions using methods that may be more accurate near 0 and 1. See ?svyciprop for alternatives:

svyciprop( ~ climate_change_concern , pew_design ,
    method = "likelihood" , na.rm = TRUE )

Regression Models and Tests of Association

Perform a design-based t-test:

svyttest( school_years ~ climate_change_concern , pew_design )

Perform a chi-squared test of association for survey data:

svychisq( 
    ~ climate_change_concern + country_economic_situation , 
    pew_design 
)

Perform a survey-weighted generalized linear model:

glm_result <- 
    svyglm( 
        school_years ~ climate_change_concern + country_economic_situation , 
        pew_design 
    )

summary( glm_result )

Analysis Examples with srvyr  

The R srvyr library calculates summary statistics from survey data, such as the mean, total or quantile using dplyr-like syntax. srvyr allows for the use of many verbs, such as summarize, group_by, and mutate, the convenience of pipe-able functions, the tidyverse style of non-standard evaluation and more consistent return types than the survey package. This vignette details the available features. As a starting point for PEW users, this code replicates previously-presented examples:

library(srvyr)
pew_srvyr_design <- as_survey( pew_design )

Calculate the mean (average) of a linear variable, overall and by groups:

pew_srvyr_design %>%
    summarize( mean = survey_mean( school_years , na.rm = TRUE ) )

pew_srvyr_design %>%
    group_by( your_day_today ) %>%
    summarize( mean = survey_mean( school_years , na.rm = TRUE ) )

Replication Example