Sampling Distribution, Central Limit Theorem, and Bootstrapping

Notes and in-class exercises

You can download the .qmd file for this activity here and open in R-studio. The rendered version is posted in the course website (Activities tab). I often experiment with the class activities (and see it in live!) and make updates, but I always post the final version before class starts. To be sure you have the most up-to-date copy, please download it once you’ve settled in before class begins.

Notes

Learning goals

Let \(\beta\) be some population parameter and \(\hat{\beta}\) be a sample estimate of \(\beta\). Our goals for the day are to:

use simulation to solidify our understanding of sampling distributions and standard errors
explore and compare two approaches to approximating the sampling distribution of \(\hat{\beta}\):
- Central Limit Theorem (CLT)
- bootstrapping
explore the impact of sample size on sampling distributions and standard errors

Readings and videos

Please watch/do the following videos and readings before class:

Reading: Section 6.7 in the STAT 155 Notes
Video 1: sampling distributions
Video 2: Central Limit Theorem
Video 3: bootstrapping

Warm-Up

Rivers contain small concentrations of mercury which can accumulate in fish. Scientists studied this phenomenon among largemouth bass in the Wacamaw and Lumber rivers of North Carolina.

One goal of this study was to explore the relationship of a fish’s mercury concentration (Concen) with its size, specifically its Length:

\[ E(Concen | Length) = \beta_0 + \beta_1 Length \]

To this end, they caught and evaluated 171 fish, and recorded the following:

variable	meaning
River	Lumber or Wacamaw
Station	Station number where the fish was caught (0, 1, …, 15)
Length	Fish’s length (in centimeters)
Weight	Fish’s weight (in grams)
Concen	Fish’s mercury concentration (in parts per million; ppm)

# Load the data & packages
library(tidyverse)
fish <- read_csv("https://Mac-STAT.github.io/data/Mercury.csv")

Plot and model the relationship of mercury concentration with length:

fish %>% 
  ggplot(aes(y = Concen, x = Length)) + 
  geom_point() + 
  geom_smooth(method = "lm", se = FALSE)


fish_model <- lm(Concen ~ Length, data = fish)
coef(summary(fish_model))
##                Estimate  Std. Error   t value     Pr(>|t|)
## (Intercept) -1.13164542 0.213614796 -5.297598 3.617750e-07
## Length       0.05812749 0.005227593 11.119359 6.641225e-22

In the summary table, is the Length coefficient 0.058 the population parameter \(\beta_1\) or a sample estimate \(\hat{\beta}_1\)?

Your response here

Since we don’t know \(\beta_1\), we can’t know the exact error in \(\hat{\beta}_1\)! This is where sampling distributions come in.

They describe how estimates \(\hat{\beta}_1\) might vary from sample to sample, thus how far these estimates might fall from \(\beta_1\):

For each of the concepts in the diagram above (Superpopulation, Finite Population, Sample), these represent for this specific data example:

Superpopulation: The true underlying process that governs the relationship between mercury concentration and length of fish, for every fish that has ever existed and ever will exist in these two rivers!

Finite Population: At the time the data was collected, the true observed relationship between mercury concentration and length of fish, for all fish in the two rivers at that time.

Sample: In our data, the observed/estimated relationship between mercury concentration and length of fish (this is \(\hat{\beta}_1\)!).

In the previous activity, each student took 1 sample of 10 observations.

This gave us a quick sense of how these estimates could vary from sample to sample.

When trying to understand/approximate our sampling distribution, there are two main approaches we can take:

When our sample size n is “large enough”, we might approximate the sampling distribution using the CLT:

\[\hat{\beta}_1 \sim N(\beta_1, \text{standard error}^2)\]

The standard error in the CLT is approximated from our sample via some formula \(c / \sqrt{n}\) where “c” is complicated.

Obtain and interpret this standard error from the model summary table:

coef(summary(fish_model))
##                Estimate  Std. Error   t value     Pr(>|t|)
## (Intercept) -1.13164542 0.213614796 -5.297598 3.617750e-07
## Length       0.05812749 0.005227593 11.119359 6.641225e-22

Our second option is something called bootstrapping. It turns out that we can actually resample from our observed data, to approximate the sampling distribution of our estimate! If it feels somewhat magical to you that this works out, that’s a very reasonable feeling.

The saying “to pull oneself up by the bootstraps” is often attributed to Rudolf Erich Raspe’s 1781 The Surprising Adventures of Baron Munchausen in which the character pulls himself out of a swamp by his hair (not bootstraps). In short, it means to get something from nothing, through your own effort:

In this spirit, statistical bootstrapping doesn’t make any probability model assumptions. It uses only the information from our one sample to approximate standard errors.

REFLECT

Great! We have two options. Here are some things to think about / reflect on:

We can approximate the sampling distribution and standard error using the CLT. BUT:
- the quality of this approximation hinges upon the validity of the Central Limit theorem which hinges upon the validity of the theoretical model assumptions, as well as a large sample size
- the CLT uses theoretical formulas for the standard error estimates, thus can feel a little mysterious without a solid foundation in probability theory
We can approxiate the sampling distribution and standard error using bootstrapping. BUT:
- it feels magical. The statistical theory behind bootstrapping is quite complicated, and there are certain obscure cases (none that we will encounter in Stat 155) where the assumptions underlying bootstrapping fail to hold

Neither approach is perfect, but they complement one another. Bootrapping in particular, while it cannot and should not replace the CLT, gives us some nice intuition behind the idea of resampling, which is fundamental for hypothesis testing (which we’ll get to shortly!).

Exercises:

Make sure that you are synchronized in your group.
Work through the following exercises in your group - match your answer with the solution
If you finish all of the exercises, then you may collect your quiz paper and enjoy the rest of the day! I will see you on Friday!!

Exercise 1: 500 samples of size 10

Recall that we can sample 10 observations from our dataset using sample_n():

# Run this chunk a few times to explore the different samples you get
fish %>% 
  sample_n(size = 10, replace = TRUE)
## # A tibble: 10 × 5
##    River   Station Length Weight Concen
##    <chr>     <dbl>  <dbl>  <dbl>  <dbl>
##  1 Wacamaw      11   36.2    708   1.3 
##  2 Lumber        1   43.5   2674   0.54
##  3 Wacamaw       9   54     2664   1.8 
##  4 Wacamaw      12   29      307   0.11
##  5 Lumber        4   27.5    308   0.94
##  6 Lumber        1   29.5    754   0.38
##  7 Lumber        0   45.2   1199   0.73
##  8 Wacamaw      10   46     1342   2.2 
##  9 Lumber        2   30      329   0.85
## 10 Wacamaw       7   30.7    315   0.44

We can take a sample and then use the data to estimate the model:

# Run this chunk a few times to explore the different sample models you get
fish %>% 
  sample_n(size = 10, replace = TRUE) %>% 
  with(lm(Concen ~ Length))
## 
## Call:
## lm(formula = Concen ~ Length)
## 
## Coefficients:
## (Intercept)       Length  
##    -0.51366      0.04035

We can also take multiple unique samples and build a sample model from each.

The code below obtains 500 separate samples of 10 fish, and stores the model estimates from each:

# Set the seed so that we all get the same results
set.seed(155)


# Store the sample models
sample_models_10 <- map_df(1:500, function(i){
    fish %>% 
    sample_n(size = 10, replace = TRUE) %>% 
    lm(Concen ~ Length, data = .) %>% 
    coef()
})

# Check it out
head(sample_models_10)
## # A tibble: 6 × 2
##   `(Intercept)` Length
##           <dbl>  <dbl>
## 1        -1.18  0.0622
## 2        -2.78  0.0964
## 3        -3.07  0.110 
## 4        -1.29  0.0593
## 5        -0.925 0.0541
## 6        -1.09  0.0535
dim(sample_models_10)
## [1] 500   2

What’s the point of the map_df() function?!? If you’ve taken any COMP classes, what process do you think `map_df() is a shortcut for?

map_df() repeats the code within the parentheses as many times as you tell it. map_df()` does repetition like a for loop.

What is stored in the Intercept and Length columns of the results?

500 different sample estimates of the model

We’ll obtain a bootstrapping distribution of \(\hat{\beta}_1\) by taking many (500, in this case) different samples of every fish in our dataset (171 of them) and exploring the degree to which \(\hat{\beta}_1\) varies from sample to sample.

Edit the code below to obtain a bootstrapping distribution.

# Set the seed so that we all get the same results
set.seed(155)

# Store the sample models
sample_models_boot <- map_df(1:___, function(i){
    fish %>% 
    sample_n(size = ___, replace = TRUE) %>% 
    lm(Concen ~ Length, data = .) %>% 
    coef()
})
## Error in parse(text = input): <text>:5:33: unexpected input
## 4: # Store the sample models
## 5: sample_models_boot <- map_df(1:__
##                                    ^

Exercise 2: Why “resampling” (`replace = TRUE`)?

Let’s wrap our minds around the idea of resampling, before coming back to our boostrapping distribution, using a small example of 5 fish:

# Define data
small_sample <- data.frame(
  id = 1:5,
  Length = c(44, 43, 54, 52, 40))

small_sample
##   id Length
## 1  1     44
## 2  2     43
## 3  3     54
## 4  4     52
## 5  5     40

This sample has a mean Length of 46.6 cm:

small_sample %>% 
  summarize(mean(Length))
##   mean(Length)
## 1         46.6

The chunk below samples 5 fish without replacement from our small_sample of 5 fish, and calculates their mean length. Run it several times. How do the sample and resulting mean change?

sample_1 <- sample_n(small_sample, size = 5, replace = FALSE)
sample_1
##   id Length
## 1  4     52
## 2  5     40
## 3  2     43
## 4  1     44
## 5  3     54

sample_1 %>% 
  summarize(mean(Length))
##   mean(Length)
## 1         46.6

Sampling our sample without replacement merely returns our original sample. Instead, resample 5 fish from our small_sample with replacement. Run it several times. What do you notice about the samples? About their mean lengths?

sample_2 <- sample_n(small_sample, size = 5, replace = TRUE)
sample_2
##   id Length
## 1  4     52
## 2  1     44
## 3  4     52
## 4  4     52
## 5  5     40

sample_2 %>% 
  summarize(mean(Length))
##   mean(Length)
## 1           48

Resampling our sample provides insight into the variability, hence potential error, in our sample estimates. (This works better when we have a sample bigger than 5!) As you observed in part b, each resample might include some fish from the original sample several times and others not at all.

Bonus Fact: Sampling with replacement also ensures that our resampled observations are independent, which we need in order for bootstrapping to “work”!

Exercise 3: Sampling distribution

Check out the resulting 500 bootstrapped sample models:

# Set the seed so that we all get the same results
set.seed(155)

# Store the sample models
sample_models_boot <- map_df(1:500, function(i){
    fish %>% 
    sample_n(size = 171, replace = TRUE) %>% 
    lm(Concen ~ Length, data = .) %>% 
    coef()
})

fish %>% 
  ggplot(aes(x = Length, y = Concen)) + 
  geom_smooth(method = "lm", se = FALSE) +
  geom_abline(data = sample_models_boot, 
              aes(intercept = `(Intercept)`, slope = Length), 
              color = "gray", size = 0.25) + 
  geom_smooth(method = "lm", color = "red", se = FALSE)

Let’s focus on the slopes of these 500 sample models.

A plot of the 500 slopes approximates the sampling distribution of the sample slopes.

sample_models_boot %>% 
  ggplot(aes(x = Length)) + 
  geom_density() + 
  geom_vline(xintercept = 0.05813, color = "red")

Describe the sampling distribution:

What’s its general shape?
Where is it roughly centered?
Roughly what’s its spread / i.e. what’s the range of estimates you observed?

Exercise 4: Standard error

For a more rigorous assessment of the spread among the sample slopes, let’s calculate their standard deviation:

sample_models_boot %>% 
  summarize(sd(Length))
## # A tibble: 1 × 1
##   `sd(Length)`
##          <dbl>
## 1      0.00569

Recall: The standard deviation of sample estimates is called a “standard error”.

It measures the typical distance of a sample estimate from the actual population value.

Compare the bootstrapped standard error to the standard error reported from our regression model (see the Std. Error column):

coef(summary(fish_model))
##                Estimate  Std. Error   t value     Pr(>|t|)
## (Intercept) -1.13164542 0.213614796 -5.297598 3.617750e-07
## Length       0.05812749 0.005227593 11.119359 6.641225e-22

Are they roughly equivalent?

Your response here

Exercise 5: Central Limit Theorem (CLT)

Recall that the CLT assumes that, so long as our sample size is “big enough”, the sampling distribution of the sample slope will be Normal.

Specifically, all possible sample slopes will vary Normally around the population slope.

Do your simulation results support this assumption? Why or why not?
Want more intuition into the CLT? Watch this video explanation using bunnies and dragons: https://www.youtube.com/watch?v=jvoxEYmQHNM

Exercise 6: Using the CLT

Let \(\hat{\beta}_1\) be an estimate of the (super)population slope parameter \(\beta_1\) calculated from a sample of 10 fish (sample_models_10).

Estimate the standard error of the slope from these resampled estimates

sample_models_10 %>% 
  summarize(sd(Length))
## # A tibble: 1 × 1
##   `sd(Length)`
##          <dbl>
## 1       0.0253

You should get a SE of roughly 0.026.

Thus, by the CLT, the sampling distribution of \(\hat{\beta}_1\) is:

\[\hat{\beta}_1 \sim N(\beta_1, 0.026^2)\]

Use this result with the 68-95-99.7 property of the Normal model to understand the potential error in a slope estimate.

There are many possible samples of 10 fish. What percent of these will produce an estimate \(\hat{\beta}_1\) that’s within 0.052, i.e. 2 standard errors, of the actual population slope \(\beta_1\)?
More than 2 standard errors from \(\beta_1\)?
More than 0.079, i.e. 3 standard errors, above \(\beta_1\)?

Exercise 7: CLT and the 68-95-99.7 Rule

Fill in the blanks below to complete some general properties assumed by the CLT:

___% of samples will produce \(\hat{\beta}_1\) estimates within 1 st. err. of \(\beta_1\)
___% of samples will produce \(\hat{\beta}_1\) estimates within 2 st. err. of \(\beta_1\)
___% of samples will produce \(\hat{\beta}_1\) estimates within 3 st. err. of \(\beta_1\)

Exercise 8: Increasing sample size

Now that we trust bootstrapping simulations to provide reasonable insight into sampling distributions (they agree with the CLT!), let’s use them to explore the impact of sample size on the quality of our sample estimates. Recall our different (bootstrap) sample estimates when we started with a sample of 171 fish:

# All bootstrap sample models
fish %>% 
  ggplot(aes(x = Length, y = Concen)) + 
  geom_smooth(method = "lm", se = FALSE) +
  geom_abline(data = sample_models_boot, 
              aes(intercept = `(Intercept)`, slope = Length), 
              color = "gray", size = 0.25) + 
  geom_smooth(method = "lm", color = "red", se = FALSE)

# Slopes of all bootstrap sample models
sample_models_boot %>% 
  ggplot(aes(x = Length)) + 
  geom_density()

Suppose that instead of starting with n = 171 fish, we only had a sample of n = 50 or n = 20 fish! What impact do you anticipate this having on our sample estimates:

If we had a smaller sample, how would it impact the sample model lines (top plot): Do you expect there to be more or less variability among the sample model lines?
If we had a smaller sample, how would it impact the sampling distribution of the sample slopes (bottom plot):
- Around what value to you expect it be centered?
- What general shape do you expect it to have?
- Do you expect it to be narrower (with smaller standard error) or wider (with larger standard error)?

Exercise 9: 500 samples of size n

Let’s decrease the sample size in our simulation. First, take 500 REsamples from fish, but this time make each resample just 50 fish. Then build a sample model from each sample:

set.seed(155)
sample_models_50 <- map_df(1:500, function(i){
    fish %>% 
    sample_n(size = 50, replace = TRUE) %>% 
    lm(Concen ~ Length, data = .) %>% 
    coef()
})

# Check it out
# Make sure that your first Intercept estimate is -1.9990216    
head(sample_models_50)
## # A tibble: 6 × 2
##   `(Intercept)` Length
##           <dbl>  <dbl>
## 1        -2.00  0.0795
## 2        -1.16  0.0589
## 3        -2.05  0.0811
## 4        -1.61  0.0697
## 5        -1.11  0.0542
## 6        -0.563 0.0423

Similarly, take 500 REsamples of size 20 from fish, then build a sample model from each sample:

set.seed(155)
sample_models_20 <- map_df(1:500, function(i){
    fish %>% 
    sample_n(size = 20, replace = TRUE) %>% 
    lm(Concen ~ Length, data = .) %>% 
    coef()
})

# Check it out
# Make sure that your first Intercept estimate is -2.2383596    
head(sample_models_20)
## # A tibble: 6 × 2
##   `(Intercept)` Length
##           <dbl>  <dbl>
## 1        -2.24  0.0847
## 2        -2.40  0.0884
## 3        -1.09  0.0559
## 4        -0.565 0.0431
## 5        -1.50  0.0696
## 6        -2.29  0.0894

Exercise 10: Impact of sample size (part I)

Use the 3 plots below to compare and contrast the 500 sets of sample models when using samples of size 20, 50, and 171. What happens as we increase sample size?! Was this what you expected?

# 500 sample models using samples of size 20
fish %>% 
  ggplot(aes(x = Length, y = Concen)) + 
  geom_smooth(method = "lm", se = FALSE) + 
  geom_abline(data = sample_models_20, 
              aes(intercept = `(Intercept)`, slope = Length), 
              color = "gray", size = 0.25) + 
  lims(x = c(25, 65), y = c(-1, 5))

# 500 sample models using samples of size 50
fish %>% 
  ggplot(aes(x = Length, y = Concen)) + 
  geom_smooth(method = "lm", se = FALSE) + 
  geom_abline(data = sample_models_50, 
              aes(intercept = `(Intercept)`, slope = Length), 
              color = "gray", size = 0.25) + 
  lims(x = c(25, 65), y = c(-1, 5))

# 500 sample models using samples of size 171
fish %>% 
  ggplot(aes(x = Length, y = Concen)) + 
  geom_smooth(method = "lm", se = FALSE) + 
  geom_abline(data = sample_models_boot, 
              aes(intercept = `(Intercept)`, slope = Length), 
              color = "gray", size = 0.25) + 
  lims(x = c(25, 65), y = c(-1, 5))

Exercise 11: Impact of sample size (part II)

Using the 3 plots below, let’s focus on just the sampling distributions of our 500 slope estimates \(\hat{\beta}_1\), i.e. the slopes of the lines in the above plots. How do the shapes, centers, and spreads of these sampling distributions compare? Was this what you expected?

# 500 sample slopes using samples of size 20
sample_models_20 %>% 
  ggplot(aes(x = Length)) + 
  geom_density() +
  lims(x = c(-0.01, 0.13), y = c(0, 70))

# 500 sample slopes using samples of size 50
sample_models_50 %>% 
  ggplot(aes(x = Length)) + 
  geom_density() +
  lims(x = c(-0.01, 0.13), y = c(0, 70))

# 500 sample slopes using samples of size 171
sample_models_boot %>% 
  ggplot(aes(x = Length)) + 
  geom_density() +
  lims(x = c(-0.01, 0.13), y = c(0, 70))

Exercise 12: Properties of sampling distributions

In light of your observations, complete the following statements about the sampling distribution of the sample slope.

For all sample sizes, the shape of the sampling distribution is roughly ___ and the sampling distribution is roughly centered around ___, the sample estimate from our original data.
As sample size increases:
The average sample slope estimate INCREASES / DECREASES / IS FAIRLY STABLE.
The standard error of the sample slopes INCREASES / DECREASES / IS FAIRLY STABLE.
Thus, as sample size increases, our sample slopes become MORE RELIABLE / LESS RELIABLE.

Solutions

Exercise 1: 500 samples of size 10

map_df() repeats the code within the parentheses as many times as you tell it. map_df()` does repetition like a for loop.
500 different sample estimates of the model

# Set the seed so that we all get the same results
set.seed(155)

# Store the sample models
sample_models_boot <- map_df(1:500, function(i){
    fish %>% 
    sample_n(size = 171, replace = TRUE) %>% 
    lm(Concen ~ Length, data = .) %>% 
    coef()
})

Exercise 2: Why “resampling” (`replace = TRUE`)?

The sample and the mean are the same every time!
If we rerun the code below multiple times, we’ll get different samples every time! Note that some of the observations are repeated (this is because of replace = TRUE), but we actually obtain variation in our samples and their mean lengths.

sample_2 <- sample_n(small_sample, size = 5, replace = TRUE)
sample_2
##   id Length
## 1  5     40
## 2  5     40
## 3  3     54
## 4  3     54
## 5  3     54

sample_2 %>% 
  summarize(mean(Length))
##   mean(Length)
## 1         48.4

Exercise 3: Sampling distribution

fish %>% 
  ggplot(aes(x = Length, y = Concen)) + 
  geom_smooth(method = "lm", se = FALSE) +
  geom_abline(data = sample_models_boot, 
              aes(intercept = Intercept, slope = Length), 
              color = "gray", size = 0.25) + 
  geom_smooth(method = "lm", color = "red", se = FALSE)
## Error in `geom_abline()`:
## ! Problem while computing aesthetics.
## ℹ Error occurred in the 2nd layer.
## Caused by error:
## ! object 'Intercept' not found

sample_models_boot %>% 
  ggplot(aes(x = Length)) + 
  geom_density() + 
  geom_vline(xintercept = 0.05813, color = "red")

The sampling distribution is symmetric, unimodal, and shaped like a bell curve!
It is roughly centered at the slope calculated from our entire sample!
Most of the estimates lie within the range 0.04 to 0.075.

Exercise 5: Standard error

# boostrapped se
sample_models_boot %>% 
  summarize(sd(Length))
## # A tibble: 1 × 1
##   `sd(Length)`
##          <dbl>
## 1      0.00569

# CLT se
coef(summary(fish_model))
##                Estimate  Std. Error   t value     Pr(>|t|)
## (Intercept) -1.13164542 0.213614796 -5.297598 3.617750e-07
## Length       0.05812749 0.005227593 11.119359 6.641225e-22

They are basically identical! Both are about 0.005.

Exercise 5: Central Limit Theorem (CLT)

Recall that the CLT assumes that, so long as our sample size is “big enough”, the sampling distribution of the sample slope will be Normal.

Specifically, all possible sample slopes will vary Normally around the population slope.

Do your simulation results support this assumption? Why or why not?

Yes! They support this assumption because the shape of sampling distribution is roughly normal (i.e. bell-shaped).

Exercise 6: Using the CLT

# Hint: Adapt the code from Exercise 5...
sample_models_10 %>% 
  summarize(sd(Length))
##   sd(Length)
## 1          0

95%
100% - 95% = 5%
(100 - 99.7)/2 = 0.15% (Note that we divide by two here, because we only want those above 3 SEs, not either above or below!)

Exercise 7: CLT and the 68-95-99.7 Rule

68% of samples will produce \(\hat{\beta}_1\) estimates within 1 st. err. of \(\beta_1\)
95% of samples will produce \(\hat{\beta}_1\) estimates within 2 st. err. of \(\beta_1\)
99.7% of samples will produce \(\hat{\beta}_1\) estimates within 3 st. err. of \(\beta_1\)

Exercise 8: Increasing sample size

Intuition, no wrong answer.

Exercise 9: 500 samples of size n

set.seed(155)
sample_models_50 <- map_df(1:500, function(i){
    fish %>% 
    sample_n(size = 50, replace = TRUE) %>% 
    lm(Concen ~ Length, data = .) %>% 
    coef()
})

# Check it out
# Make sure that your first Intercept estimate is -1.9990216    
head(sample_models_50)
## # A tibble: 6 × 2
##   `(Intercept)` Length
##           <dbl>  <dbl>
## 1        -2.00  0.0795
## 2        -1.16  0.0589
## 3        -2.05  0.0811
## 4        -1.61  0.0697
## 5        -1.11  0.0542
## 6        -0.563 0.0423

set.seed(155)
sample_models_20 <- map_df(1:500, function(i){
    fish %>% 
    sample_n(size = 20, replace = TRUE) %>% 
    lm(Concen ~ Length, data = .) %>% 
    coef()
})

# Check it out
# Make sure that your first Intercept estimate is -2.2383596    
head(sample_models_20)
## # A tibble: 6 × 2
##   `(Intercept)` Length
##           <dbl>  <dbl>
## 1        -2.24  0.0847
## 2        -2.40  0.0884
## 3        -1.09  0.0559
## 4        -0.565 0.0431
## 5        -1.50  0.0696
## 6        -2.29  0.0894

Exercise 10: Impact of sample size (part I)

The sample model lines become less and less variable from sample to sample.

# 500 sample models using samples of size 20
fish %>% 
  ggplot(aes(x = Length, y = Concen)) + 
  geom_smooth(method = "lm", se = FALSE) + 
  geom_abline(data = sample_models_20, 
              aes(intercept = `(Intercept)`, slope = Length), 
              color = "gray", size = 0.25) + 
  lims(x = c(25, 65), y = c(-1, 5))



# 500 sample models using samples of size 50
fish %>% 
  ggplot(aes(x = Length, y = Concen)) + 
  geom_smooth(method = "lm", se = FALSE) + 
  geom_abline(data = sample_models_50, 
              aes(intercept = `(Intercept)`, slope = Length), 
              color = "gray", size = 0.25) + 
  lims(x = c(25, 65), y = c(-1, 5))



# 500 sample models using samples of size 171
fish %>% 
  ggplot(aes(x = Length, y = Concen)) + 
  geom_smooth(method = "lm", se = FALSE) + 
  geom_abline(data = sample_models_boot, 
              aes(intercept = `(Intercept)`, slope = Length), 
              color = "gray", size = 0.25) + 
  lims(x = c(25, 65), y = c(-1, 5))

Exercise 11: Impact of sample size (part II)

No matter the sample size, the sample estimates are normally distributed around the same value (here the sample slope since we’re sampling from the sample). But as sample size increases, the variability of the sample estimates decreases.

# 500 sample slopes using samples of size 20
sample_models_20 %>% 
  ggplot(aes(x = Length)) + 
  geom_density() +
  lims(x = c(-0.01, 0.13), y = c(0, 70))


# 500 sample slopes using samples of size 50
sample_models_50 %>% 
  ggplot(aes(x = Length)) + 
  geom_density() +
  lims(x = c(-0.01, 0.13), y = c(0, 70))


# 500 sample slopes using samples of size 171
sample_models_171 %>% 
  ggplot(aes(x = Length)) + 
  geom_density() +
  lims(x = c(-0.01, 0.13), y = c(0, 70))
## Error: object 'sample_models_171' not found

Exercise 12: Properties of sampling distributions

In light of your observations, complete the following statements about the sampling distribution of the sample slope.

For all sample sizes, the shape of the sampling distribution is roughly normal and the sampling distribution is roughly centered around 0.05813, the sample estimate from our original data.
As sample size increases:
The average sample slope estimate IS FAIRLY STABLE.
The standard error of the sample slopes DECREASES.
Thus, as sample size increases, our sample slopes become MORE RELIABLE.

Notes

Learning goals

Readings and videos

Warm-Up

REFLECT

Exercises:

Exercise 1: 500 samples of size 10

Exercise 2: Why “resampling” (replace = TRUE)?

Exercise 3: Sampling distribution

Exercise 4: Standard error

Exercise 5: Central Limit Theorem (CLT)

Exercise 6: Using the CLT

Exercise 7: CLT and the 68-95-99.7 Rule

Exercise 8: Increasing sample size

Exercise 9: 500 samples of size n

Exercise 10: Impact of sample size (part I)

Exercise 11: Impact of sample size (part II)

Exercise 12: Properties of sampling distributions

Solutions

Exercise 1: 500 samples of size 10

Exercise 2: Why “resampling” (replace = TRUE)?

Exercise 3: Sampling distribution

Exercise 5: Standard error

Exercise 5: Central Limit Theorem (CLT)

Exercise 6: Using the CLT

Exercise 7: CLT and the 68-95-99.7 Rule

Exercise 8: Increasing sample size

Exercise 9: 500 samples of size n

Exercise 10: Impact of sample size (part I)

Exercise 11: Impact of sample size (part II)

Exercise 12: Properties of sampling distributions

Exercise 2: Why “resampling” (`replace = TRUE`)?

Exercise 2: Why “resampling” (`replace = TRUE`)?