Lecture 7: Exercises

library(tidyverse)

Exercise 1: Hypothesis testing

Similarly to dataset mtcars, the dataset mpg from ggplot package includes data on automobiles. However, mpg includes data for newer cars from year 1999 and 2008. The variables measured for each car is slighly different. Here we are interested in the variable, hwy, the highway miles per gallon.

# We create a new culumn with manual/automatic data only
mpg <- mpg %>% 
  mutate(
    transmission = factor(
        gsub("\\((.*)", "", trans), levels = c("auto", "manual"))
  )
mpg

# A tibble: 234 x 12
   manufacturer model      displ  year   cyl trans drv     cty   hwy fl    class   transmission
   <chr>        <chr>      <dbl> <int> <int> <fct> <chr> <int> <int> <chr> <chr>   <fct>       
 1 audi         a4           1.8  1999     4 auto  f        18    29 p     compact auto        
 2 audi         a4           1.8  1999     4 auto  f        21    29 p     compact auto        
 3 audi         a4           2    2008     4 auto  f        20    31 p     compact auto        
 4 audi         a4           2    2008     4 auto  f        21    30 p     compact auto        
 5 audi         a4           2.8  1999     6 auto  f        16    26 p     compact auto        
 6 audi         a4           2.8  1999     6 auto  f        18    26 p     compact auto        
 7 audi         a4           3.1  2008     6 auto  f        18    27 p     compact auto        
 8 audi         a4 quattro   1.8  1999     4 auto  4        18    26 p     compact auto        
 9 audi         a4 quattro   1.8  1999     4 auto  4        16    25 p     compact auto        
10 audi         a4 quattro   2    2008     4 auto  4        20    28 p     compact auto        
# ... with 224 more rows

Part 1: One-sample test

1. Subset the mpg dataset to inlude only cars from year 2008.

2. Test whether cars from 2008 have mean the highway miles per gallon, hwy, equal to 30 mpg.

3. Test whether cars from 2008 with 4 cylinders have mean hwy equal to 30 mpg.

Part 2: Two-sample test

1. Test if the mean hwy for automatic is less than that for manual cars in 2008. Generate a boxplot with jittered points for hwy for each transmission group.

2. Test if the mean hwy for cars from 1999 and is greater than that for cars from 2008. Generate a boxplot with jittered points for hwy for each year group.

Exercise 2: Logistic Regression

In this you will use a dataset Default, on customer default records for a credit card company, which is included in ISL book. To obtain the data you will need to install a package ISLR.

# install.packages("ISLR")
library(ISLR)
(Default <- tbl_df(Default))  # convert data.frame to tibble

# A tibble: 10,000 x 4
   default student balance income
   <fct>   <fct>     <dbl>  <dbl>
 1 No      No         730. 44362.
 2 No      Yes        817. 12106.
 3 No      No        1074. 31767.
 4 No      No         529. 35704.
 5 No      No         786. 38463.
 6 No      Yes        920.  7492.
 7 No      No         826. 24905.
 8 No      Yes        809. 17600.
 9 No      No        1161. 37469.
10 No      No           0  29275.
# ... with 9,990 more rows

1. First, divide your dataset into a train and test set. Randomly sample 6000 observations and include them in the train set, and the remaining use as a test set.

2. Fit a logistic regression including all the features to predict whether a customer defaulted or not.

3. Note if any variables seem not significant. Then, adjust your model accordingly (by removing them).

4. Compute the predicted probabilities of ‘default’ for the observations in the test set. Then evaluate the model accuracy.

5. For the test set, generate a scatterplot of ‘balance’ vs ‘default’ with points colored by ‘student’ factor. Then, overlay a line plot of the predicted probability of default as computed in the previous question. You should plot two lines for student and non student separately by setting the ‘color = student’.

Exercise 3: Random Forest

In this exercise we will build a random forest model based on the data used to create the visualization here.

# Skip first 2 lines since they were comments
url <- paste0("https://raw.githubusercontent.com/jadeyee/r2d3-part-1-data/",
              "master/part_1_data.csv")
houses <- read_csv(url, skip = 2)
houses <- tbl_df(houses)
houses <- houses %>%
    mutate(city = factor(in_sf, levels = c(1, 0), labels = c("SF", "NYC")))
houses 

# A tibble: 492 x 9
   in_sf  beds  bath   price year_built  sqft price_per_sqft elevation city 
   <int> <dbl> <dbl>   <int>      <int> <int>          <int>     <int> <fct>
 1     0     2     1  999000       1960  1000            999        10 NYC  
 2     0     2     2 2750000       2006  1418           1939         0 NYC  
 3     0     2     2 1350000       1900  2150            628         9 NYC  
 4     0     1     1  629000       1903   500           1258         9 NYC  
 5     0     0     1  439000       1930   500            878        10 NYC  
 6     0     0     1  439000       1930   500            878        10 NYC  
 7     0     1     1  475000       1920   500            950        10 NYC  
 8     0     1     1  975000       1930   900           1083        10 NYC  
 9     0     1     1  975000       1930   900           1083        12 NYC  
10     0     2     1 1895000       1921  1000           1895        12 NYC  
# ... with 482 more rows

1. Using pairs() function plot the relationship between every variable pairs. You can color the points by the city the observation corresponds to; set the color argument in pairs() as follows: col = houses$in_sf + 3L

2. Split the data into (70%-30%) train and test set. How many observations are in your train and test sets?

3. Train a random forest on the train set, using all the variables in the model, to classify houses into the ones from San Francisco and from New York. Remember to remove ‘in_sf’, as it is the same variable as ‘city’.

4. Compute predictions and print out the confusion (error) matrix for the test set to asses the model accuracy. Also, compute the model accuracy.

5. Which features were the most predictive for classifying houses into SF vs NYC groups? Use importance measures to answer the question.