Pesquisa Nacional por Amostra de Domicilios (SIPP)

Build Status Build status

Contributed by Dr. Djalma Pessoa <pessoad@gmail.com>

Brazil’s previous principal household survey, the Pesquisa Nacional por Amostra de Domicilios (PNAD) measures general education, labor, income, and housing characteristics of the population.

  • One table with one row per sampled household and a second table with one row per individual within each sampled household.

  • A complex sample survey designed to generalize to the civilian non-institutional population of Brazil, although the rural north was not included prior to 2004.

  • Released annually since 2001 except for years ending in zero, when the decennial census takes its place.

  • Administered by the Instituto Brasileiro de Geografia e Estatistica.

Simplified Download and Importation

The R lodown package easily downloads and imports all available SIPP microdata by simply specifying "sipp" with an output_dir = parameter in the lodown() function. Depending on your internet connection and computer processing speed, you might prefer to run this step overnight.

library(lodown)
lodown( "sipp" , output_dir = file.path( path.expand( "~" ) , "SIPP" ) )

lodown also provides a catalog of available microdata extracts with the get_catalog() function. After requesting the SIPP catalog, you could pass a subsetted catalog through the lodown() function in order to download and import specific extracts (rather than all available extracts).

library(lodown)
# examine all available SIPP microdata files
sipp_cat <-
    get_catalog( "sipp" ,
        output_dir = file.path( path.expand( "~" ) , "SIPP" ) )

# 2011 only
sipp_cat <- subset( sipp_cat , year == 2011 )
# download the microdata to your local computer
sipp_cat <- lodown( "sipp" , sipp_cat )

Analysis Examples with the survey library  

Construct a database-backed complex sample survey design:

library(DBI)
library(RSQLite)
library(survey)

options( survey.lonely.psu = "adjust" )

prestratified_design <-
    svydesign(
        id = ~v4618 ,
        strata = ~v4617 ,
        data = sipp_cat[ 1 , "db_tablename" ] ,
        weights = ~pre_wgt ,
        nest = TRUE ,
        dbtype = "SQLite" ,
        dbname = sipp_cat[ 1 , "dbfile" ]
    )
    
sipp_design <- 
    lodown:::pnad_postStratify( 
        design = prestratified_design ,
        strata.col = 'v4609' ,
        oldwgt = 'pre_wgt'
    )

Variable Recoding

Add new columns to the data set:

sipp_design <- 
    update( 
        sipp_design , 
        age_categories = factor( 1 + findInterval( v8005 , seq( 5 , 60 , 5 ) ) ) ,
        male = as.numeric( v0302 == 2 ) ,
        teenagers = as.numeric( v8005 > 12 & v8005 < 20 ) ,
        started_working_before_thirteen = as.numeric( v9892 < 13 )
    )

Unweighted Counts

Count the unweighted number of records in the survey sample, overall and by groups:

sum( weights( sipp_design , "sampling" ) != 0 )

svyby( ~ one , ~ region , sipp_design , unwtd.count )

Weighted Counts

Count the weighted size of the generalizable population, overall and by groups:

svytotal( ~ one , sipp_design )

svyby( ~ one , ~ region , sipp_design , svytotal )

Descriptive Statistics

Calculate the mean (average) of a linear variable, overall and by groups:

svymean( ~ v4720 , sipp_design , na.rm = TRUE )

svyby( ~ v4720 , ~ region , sipp_design , svymean , na.rm = TRUE )

Calculate the distribution of a categorical variable, overall and by groups:

svymean( ~ age_categories , sipp_design )

svyby( ~ age_categories , ~ region , sipp_design , svymean )

Calculate the sum of a linear variable, overall and by groups:

svytotal( ~ v4720 , sipp_design , na.rm = TRUE )

svyby( ~ v4720 , ~ region , sipp_design , svytotal , na.rm = TRUE )

Calculate the weighted sum of a categorical variable, overall and by groups:

svytotal( ~ age_categories , sipp_design )

svyby( ~ age_categories , ~ region , sipp_design , svytotal )

Calculate the median (50th percentile) of a linear variable, overall and by groups:

svyquantile( ~ v4720 , sipp_design , 0.5 , na.rm = TRUE )

svyby( 
    ~ v4720 , 
    ~ region , 
    sipp_design , 
    svyquantile , 
    0.5 ,
    ci = TRUE ,
    keep.var = TRUE ,
    na.rm = TRUE
)

Estimate a ratio:

svyratio( 
    numerator = ~ started_working_before_thirteen , 
    denominator = ~ teenagers , 
    sipp_design ,
    na.rm = TRUE
)

Subsetting

Restrict the survey design to married persons:

sub_sipp_design <- subset( sipp_design , v4011 == 1 )

Calculate the mean (average) of this subset:

svymean( ~ v4720 , sub_sipp_design , na.rm = TRUE )

Measures of Uncertainty

Extract the coefficient, standard error, confidence interval, and coefficient of variation from any descriptive statistics function result, overall and by groups:

this_result <- svymean( ~ v4720 , sipp_design , na.rm = TRUE )

coef( this_result )
SE( this_result )
confint( this_result )
cv( this_result )

grouped_result <-
    svyby( 
        ~ v4720 , 
        ~ region , 
        sipp_design , 
        svymean ,
        na.rm = TRUE 
    )
    
coef( grouped_result )
SE( grouped_result )
confint( grouped_result )
cv( grouped_result )

Calculate the degrees of freedom of any survey design object:

degf( sipp_design )

Calculate the complex sample survey-adjusted variance of any statistic:

svyvar( ~ v4720 , sipp_design , na.rm = TRUE )

Include the complex sample design effect in the result for a specific statistic:

# SRS without replacement
svymean( ~ v4720 , sipp_design , na.rm = TRUE , deff = TRUE )

# SRS with replacement
svymean( ~ v4720 , sipp_design , na.rm = TRUE , deff = "replace" )

Compute confidence intervals for proportions using methods that may be more accurate near 0 and 1. See ?svyciprop for alternatives:

svyciprop( ~ male , sipp_design ,
    method = "likelihood" )

Regression Models and Tests of Association

Perform a design-based t-test:

svyttest( v4720 ~ male , sipp_design )

Perform a chi-squared test of association for survey data:

svychisq( 
    ~ male + age_categories , 
    sipp_design 
)

Perform a survey-weighted generalized linear model:

glm_result <- 
    svyglm( 
        v4720 ~ male + age_categories , 
        sipp_design 
    )

summary( glm_result )

Poverty and Inequality Estimation with convey  

The R convey library estimates measures of income concentration, poverty, inequality, and wellbeing. This textbook details the available features. As a starting point for SIPP users, this code calculates the gini coefficient on complex sample survey data:

library(convey)
sipp_design <- convey_prep( sipp_design )

sub_sipp_design <- 
    subset( 
        sipp_design , 
        !is.na( v4720 ) & v4720 != 0 & v8005 >= 15
    )

svygini( ~ v4720 , sub_sipp_design , na.rm = TRUE )

Replication Example

svytotal( ~one , sipp_design )
svytotal( ~factor( v0302 ) , sipp_design )
cv( svytotal( ~factor( v0302 ) , sipp_design ) )