New York City Housing and Vacancy Survey (NYCHVS)

Build Status Build status

The New York City Housing and Vacancy Survey (NYCHVS) covers the city-wide rental vacancy rate and other characteristics like neighborhood housing stock.

  • One table with one record per occupied housing unit, a second table with one record per person inside each occupied housing unit, and a third table with one record per unoccupied housing unit.

  • A complex sample survey designed to generalize to all occupied and unoccupied housing units in the five boroughs of New York City.

  • Released triennially since 1998.

  • Funded by the New York City Department of Housing Preservation and Development and conducted by the United States Census Bureau.

Simplified Download and Importation

The R lodown package easily downloads and imports all available NYCHVS microdata by simply specifying "nychvs" with an output_dir = parameter in the lodown() function. Depending on your internet connection and computer processing speed, you might prefer to run this step overnight.

library(lodown)
lodown( "nychvs" , output_dir = file.path( path.expand( "~" ) , "NYCHVS" ) )

lodown also provides a catalog of available microdata extracts with the get_catalog() function. After requesting the NYCHVS catalog, you could pass a subsetted catalog through the lodown() function in order to download and import specific extracts (rather than all available extracts).

library(lodown)
# examine all available NYCHVS microdata files
nychvs_cat <-
    get_catalog( "nychvs" ,
        output_dir = file.path( path.expand( "~" ) , "NYCHVS" ) )

# 2014 only
nychvs_cat <- subset( nychvs_cat , year == 2014 )
# download the microdata to your local computer
nychvs_cat <- lodown( "nychvs" , nychvs_cat )

Analysis Examples with the survey library  

Construct a complex sample survey design:

options( survey.lonely.psu = "adjust" )

library(survey)

# load the occupied units table
nychvs_df <- readRDS( file.path( path.expand( "~" ) , "NYCHVS" , "2014/occ.rds" ) )

nychvs_design <- 
    svydesign( ~ 1 , data = nychvs_df , weights = ~ fw )

Variable Recoding

Add new columns to the data set:

nychvs_design <- 
    update( 
        nychvs_design , 
        
        one = 1 ,
        
        home_owners = as.numeric( sc115 == 1 ) ,

        yearly_household_income = ifelse( uf42 == 9999999 , 0 , as.numeric( uf42 ) ) ,
        
        gross_monthly_rent = ifelse( uf17 == 99999 , NA , as.numeric( uf17 ) ) ,
        
        borough =
            factor( boro , levels = 1:5 , labels =
                c( 'Bronx' , 'Brooklyn' , 'Manhattan' , 
                'Queens' , 'Staten Island' )
            ) ,
            
        householder_sex = factor( hhr2 , labels = c( 'male' , 'female' ) )
            
    )

Unweighted Counts

Count the unweighted number of records in the survey sample, overall and by groups:

sum( weights( nychvs_design , "sampling" ) != 0 )

svyby( ~ one , ~ borough , nychvs_design , unwtd.count )

Weighted Counts

Count the weighted size of the generalizable population, overall and by groups:

svytotal( ~ one , nychvs_design )

svyby( ~ one , ~ borough , nychvs_design , svytotal )

Descriptive Statistics

Calculate the mean (average) of a linear variable, overall and by groups:

svymean( ~ yearly_household_income , nychvs_design , na.rm = TRUE )

svyby( ~ yearly_household_income , ~ borough , nychvs_design , svymean , na.rm = TRUE )

Calculate the distribution of a categorical variable, overall and by groups:

svymean( ~ householder_sex , nychvs_design )

svyby( ~ householder_sex , ~ borough , nychvs_design , svymean )

Calculate the sum of a linear variable, overall and by groups:

svytotal( ~ yearly_household_income , nychvs_design , na.rm = TRUE )

svyby( ~ yearly_household_income , ~ borough , nychvs_design , svytotal , na.rm = TRUE )

Calculate the weighted sum of a categorical variable, overall and by groups:

svytotal( ~ householder_sex , nychvs_design )

svyby( ~ householder_sex , ~ borough , nychvs_design , svytotal )

Calculate the median (50th percentile) of a linear variable, overall and by groups:

svyquantile( ~ yearly_household_income , nychvs_design , 0.5 , na.rm = TRUE )

svyby( 
    ~ yearly_household_income , 
    ~ borough , 
    nychvs_design , 
    svyquantile , 
    0.5 ,
    ci = TRUE ,
    keep.var = TRUE ,
    na.rm = TRUE
)

Estimate a ratio:

svyratio( 
    numerator = ~ gross_monthly_rent , 
    denominator = ~ yearly_household_income , 
    nychvs_design ,
    na.rm = TRUE
)

Subsetting

Restrict the survey design to Manhattan:

sub_nychvs_design <- subset( nychvs_design , boro == 3 )

Calculate the mean (average) of this subset:

svymean( ~ yearly_household_income , sub_nychvs_design , na.rm = TRUE )

Measures of Uncertainty

Extract the coefficient, standard error, confidence interval, and coefficient of variation from any descriptive statistics function result, overall and by groups:

this_result <- svymean( ~ yearly_household_income , nychvs_design , na.rm = TRUE )

coef( this_result )
SE( this_result )
confint( this_result )
cv( this_result )

grouped_result <-
    svyby( 
        ~ yearly_household_income , 
        ~ borough , 
        nychvs_design , 
        svymean ,
        na.rm = TRUE 
    )
    
coef( grouped_result )
SE( grouped_result )
confint( grouped_result )
cv( grouped_result )

Calculate the degrees of freedom of any survey design object:

degf( nychvs_design )

Calculate the complex sample survey-adjusted variance of any statistic:

svyvar( ~ yearly_household_income , nychvs_design , na.rm = TRUE )

Include the complex sample design effect in the result for a specific statistic:

# SRS without replacement
svymean( ~ yearly_household_income , nychvs_design , na.rm = TRUE , deff = TRUE )

# SRS with replacement
svymean( ~ yearly_household_income , nychvs_design , na.rm = TRUE , deff = "replace" )

Compute confidence intervals for proportions using methods that may be more accurate near 0 and 1. See ?svyciprop for alternatives:

svyciprop( ~ home_owners , nychvs_design ,
    method = "likelihood" )

Regression Models and Tests of Association

Perform a design-based t-test:

svyttest( yearly_household_income ~ home_owners , nychvs_design )

Perform a chi-squared test of association for survey data:

svychisq( 
    ~ home_owners + householder_sex , 
    nychvs_design 
)

Perform a survey-weighted generalized linear model:

glm_result <- 
    svyglm( 
        yearly_household_income ~ home_owners + householder_sex , 
        nychvs_design 
    )

summary( glm_result )

Analysis Examples with srvyr  

The R srvyr library calculates summary statistics from survey data, such as the mean, total or quantile using dplyr-like syntax. srvyr allows for the use of many verbs, such as summarize, group_by, and mutate, the convenience of pipe-able functions, the tidyverse style of non-standard evaluation and more consistent return types than the survey package. This vignette details the available features. As a starting point for NYCHVS users, this code replicates previously-presented examples:

library(srvyr)
nychvs_srvyr_design <- as_survey( nychvs_design )

Calculate the mean (average) of a linear variable, overall and by groups:

nychvs_srvyr_design %>%
    summarize( mean = survey_mean( yearly_household_income , na.rm = TRUE ) )

nychvs_srvyr_design %>%
    group_by( borough ) %>%
    summarize( mean = survey_mean( yearly_household_income , na.rm = TRUE ) )

Replication Example