Collecting and Summarizing Data

Notes and in-class exercises

You can download the .qmd file for this activity here and open in R-studio. The rendered version is posted in the course website (Activities tab). I often experiment with the class activities (and see it in live!) and make updates, but I always post the final version before class starts. To be sure you have the most up-to-date copy, please download it once you’ve settled in before class begins.

Welcome

Settling in

[We need to change the setup of the classroom (unfortunately!). Let us start an unintended class activity of making pods (by joining two tables and 4 chairs face-to-face)]
Sit in groups of 4 (preferably) or 3 (at least). Your group should include:
- nobody that you already know
- at least 1 person who has used RStudio before
Meet the people at your table. Share your names and pronouns (again!?). Discuss a high point of your summer break.

Open the online manual:
- https://mutasim221b.github.io/Mac-STAT-155-Fall-25/ (also linked in Moodle)
- Top bar < Activities < Collecting and Summarizing Data
- We will do some reading, live-note taking, in-class activities, solving examples, doing exercises…until we ran out of time!

If you have applied/approved for the waitlist, be sure to approach me after the class & register for the course today. At that point you will be added to Moodle.

Statistical Modeling?!

Statistical Modeling is the art and science of turning data into information about relationships of interest. For example, the following are just a few Mac faculty / offices that use statistical models to study relationships:

Michelle Tong (BIOL): Making “Good” Choices: Social Isolation in Mice Exacerbates the Effects of Chronic Stress on Decision Making
Ariel James (PSYCH): Language Experience Predicts Eye Movements During Online Auditory Comprehension
Sarah West (ECON): Redevelopment along arterial streets: The effects of light rail on land use change
Admissions uses data to help understand potential enrollments
Athletics uses data (eg: to better understand the relationship between sleep and outcomes)

This class is designed for you.

STAT 155 is a modern, non-traditional introduction to statistics. We’ll explore sophisticated tools that typically aren’t covered until a second course in statistics.
- This means:
  - Non-majors taking this as a terminal course will take away highly applicable and marketable knowledge & skills.
  - Majors will gain a solid foundation from which to study more advanced models & theory.
- This does NOT mean that we’re skipping a course! STAT 155 teaches introductory statistics content, but through a different lens than a traditional course (regression).
Thriving in STAT 155 is NOT correlated with the following: your major, whether you think you’re a “math person,” whether you have any previous idea what “statistical modeling” is, etc. It IS correlated with effort (time, practice, studying, completing assignments without relying on AI) and engagement (attendance, attention, collaboration).
STAT 155 emphasizes statistical applications and intuition over theory (and memorizing formulas). To focus on applications and intuition, we’ll utilize statistical software (R/RStudio). It’s assumed that you are totally new to RStudio! More on this later…
Even though I have been teaching Statistics since 2018 (as GTI!), this is the first time I am teaching this course at Mac! Please let me know as soon as you see any broken links/typos/etc., or you have any question/problem/concern.

Introductions & Data Principles

Links to related reading(s):

See class notes

Go to your class-notes to see what we covered in live-note taking!

EXAMPLE 1: Tidy data (Class Activities!)

Welcome to the first in-class activities of STAT 155! Fill out the Day 1 Activities form of the following questions (anonymous). We’ll come back to this in a future class. Wait until everyone in your group is done with this.

How many hours of sleep did you get last night?
How many cups of coffee did you drink this morning?
What is your declared or potential major? (If you are a double major, just pick whichever one you think of first.)
What is your anticipated graduation year?
How many stats/data science courses have you taken in the past?
On a scale of 1 (get me out of here) to 10 (yay!), how excited are you about this course?
Is it your birthday this semester? (yes/no)
How many unread emails do you have in your inbox right now?

R and RStudio

MOTIVATION

“Doing” statistical modeling and working with data in general requires statistical software – calculators, spreadsheet functionality, etc don’t cut it. We’ll exclusively use R and RStudio:

Figure 1.1 from A ModernDive into R and the Tidyverse.

Why R/RStudio?

it’s free
it’s open source (the code is free & anybody can contribute to it)
it has a huge online community (which is helpful for when you get stuck)
it’s an industry standard
it can be used to create reproducible and lovely documents (including this online manual!)
Fun fact: it was started by Mac alum JJ Allaire and beta-tested at Mac!

IMPORTANT: RStudio is NOT the point of this course!!

RStudio = a hammer
- Simply a tool needed for statistical modeling that you’ll learn through lots of practice, trial, and error.
- Alone, it’s not very interesting.
You = a carpenter
- You will develop the knowledge about designing statistical analyses that are useful and correct.
- You will learn to build these analyses with the appropriate tools (RStudio).
- Your analyses, not use of RStudio, are the interesting part!
You’ll pick up the RStudio basics needed for introductory statistical models. To learn more about RStudio more generally you should take COMP/STAT 112.

DIRECTIONS

If you haven’t already installed R and RStudio in your laptop, please go to Mac’s RStudio server: https://rstudio.macalester.edu/
- Sign in with your Mac username (eg: mbillah) and password.
- NOTE: After class, you’ll install R/RStudio on your own machine and will not be using the server. Please meet me in-person to know why you can’t rely on the server (specially for this course!)

Example 1: Use R as a calculator

Type the following lines in the console (bottom left), one by one, hitting Return/Enter after each line. In some cases you might even get an error! This error is important to learning how R code does and doesn’t work.

4 + 2

4^2

4*2

4(2)

Example 2: Functions and arguments

We can also use built-in functions to perform common tasks. These functions have names and require information about arguments in order to run:

function(argument) Cheatcode: RiceCooker(Rice)

Try out the following functions one by one in the RStudio console. For each function, note its…

name
the argument or information it needs to run
what output it produces (what the function does)
how the name connects to what the function does

sqrt(9)

nchar("macalester")

sqrt(nchar("snow"))

Some functions have more than 1 argument, separated by commas:

function(argument1 = ___, argument2 = ___) Cheatcode: RiceCooker(Rice,Chicken)

Try out the following, one by one.

rep(x = 2, times = 5)

rep(times = 5, x = 2)

rep(2, 5)

rep(5, 2)

Finally, R is case sensitive. Try using Rep() instead of rep(). Take time to read the error message!

Rep(5, 2)

Example 3: Save it for later

We’ll often want to store some R output for later use. In R:

name <- output

where name is the name under which to store a result, output is the result we wish to store, and <- is the assignment operator (I think of this as an arrow pointing the output into the name).

IMPORTANT: Try out each line one at a time. Why doesn’t the first line produce any output?

degrees_c <- -13

degrees_c

degrees_c * (9/5) + 32

Example 4: Import data

Next, let’s work with some data!! The first step is importing our data into RStudio. How we do this depends on:

file format (eg: .xls Excel spreadsheet, .csv, .txt)
file location (eg: online, on your desktop, built into RStudio itself).

The data from the survey you took before class is stored as a .csv file online. Import this data using the read_csv() function, and store it as survey using the code below:

First, in the Console pane of RStudio, run the following command to install some necessary packages (you will need to do this any time you are installing a new package):

install.packages("tidyverse")

# Load the "tidyverse" package which contains the read_csv() function
library(tidyverse)

# Import the data
survey <- read_csv("https://mac-stat.github.io/data/112_fall_2024_survey.csv")

Check out the data

In the Environment tab in the upper right pane of RStudio, click on survey. What happens?!

In the modern era, datasets often contain hundreds of variables and millions of observations. We need more effective ways to explore such data.

Example 5: Get to know the data

PAUSE: Make sure you’re still in sync with your group.

Before we can learn anything from our data, we must understand its structure. For each function below:

try it out
discuss with your group what the function does
discuss with your group how the function’s name connects to what it does

dim(survey) # (Number of row (case/obs.), Number of column (variable))

nrow(survey) # Number of case/obs.

ncol(survey) # Number of variables

head(survey) # View first few rows of the dataset (6 rows, by default)

head(survey, 3) # Controlling the view of first few rows of the dataset

tail(survey) # View first few rows of the dataset (6 rows, by default)

names(survey) # Get all column (variable) names

str(survey) # Overall info about data

Example 6 : Code = communication

It’s important to recognize from day 1 that code is a form of communication, both to yourself and others!!!!! Code structure and details are important to readability and clarity, just as grammar, punctuation, spelling, paragraphs, and line spacing are important in written essays. All of the code below works, but has bad structure. With your group, discuss what is unfortunate about each line, then make it better.

seq(from=1, to=9, by=2)
seq(from = 1, to=9, by=2)
temp_cel <- -13
thisisthetemperaturetodayincelsius <- -13
this_is_the_temperature_today_in_celsius <- -13

Example 7: You will make so many mistakes!

Mistakes are common when, and even important to, learning any new language. You’ll get better and better at interpreting error messages, finding help, and fixing errors. In addition to finding help online, R has built-in help files. For example:

In the console, type ?rep and press Return/Enter.
Check out the documentation file that pops up in the Help tab (lower right).
Quickly scroll through, noting the type of information provided.
Pause at the “Examples” section at the bottom – perhaps the most useful section! Try out a couple of the provided examples in your console.

Example 8: Make a “cheat sheet”

You will continue to pick up new R code and ideas. You’re highly encouraged to start tracking this in a cheat sheet (eg: in a Google doc). The cheat sheet will be a handy reference for you, and the act of making it will help deepen your understanding and retention.

Exercise: Complete this after the class

Complete this exercise after class. First, try it on your own (or with your group), and then check your work against the solution provided at the end of this .qmd file.

Use R code to do the following:

Import & name data on different Himalayan peaks from the url below:
https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-09-22/peaks.csv NOTE: A codebook, i.e. a description of the data, is here.
Use a function to show which variables are recorded on each peak.
How many peaks are included in the dataset? Answer this using a function, not by counting up the rows yourself.
Show the first 6 rows of the dataset. NOTE: This gives us a quick glimpse without having to print out the entire dataset!

We start Univariate Visualization and Summaries activities now

Solutions

Exercise 1: Use R as a calculator

4 + 2
## [1] 6
4^2
## [1] 16
4*2
## [1] 8
#4(2) # We need to use * for multiplication

Exercise 2: Functions and arguments

# Calculate the square root of 9
sqrt(9)
## [1] 3

# Calculate the number of characters in the word "macalester"
nchar("macalester")
## [1] 10

# Calculate the square root of the number of characters in the word "snow"
sqrt(nchar("snow"))
## [1] 2

# Repeat the number 2, 5 times
rep(x = 2, times = 5)
## [1] 2 2 2 2 2

# Repeat the number 2, 5 times
rep(times = 5, x = 2)
## [1] 2 2 2 2 2

# Repeat the number 2, 5 times
rep(2, 5)
## [1] 2 2 2 2 2

# Repeat the number 5, 2 times
rep(5, 2)
## [1] 5 5

Exercise 3: Save it for later

# Nothing shows up -- all we're doing here is storing -13 as degrees_c
degrees_c <- -13

# Print the contents of degrees_c
degrees_c
## [1] -13

# We can "do math" with the contents of degrees_c
degrees_c * (9/5) + 32
## [1] 8.6

Exercise 4: Import data

# Load the tidyverse package
library(tidyverse)

# Import the data
survey <- read_csv("https://mac-stat.github.io/data/112_fall_2024_survey.csv")

Exercise 5: Get to know the data

# Dimensions of the survey data set
# First number = number of rows
# Second number = number of columns
dim(survey)
## [1] 98  4

# Number of rows in the survey data set
nrow(survey)
## [1] 98

# First 6 rows (the head) of the survey data set
head(survey)
## # A tibble: 6 × 4
##   cafe_mac     minutes_to_campus fave_temp hangout      
##   <chr>                    <dbl>     <dbl> <chr>        
## 1 Cheesecake                  15        18 the mountains
## 2 Cheese pizza                10        24 a beach      
## 3 udon noodles                 4        18 the mountains
## 4 egg rolls                    7        10 a beach      
## 5 Tacos                        5        18 the mountains
## 6 pasta                       35         7 the mountains

# First 3 rows of the survey data set
head(survey, 3)
## # A tibble: 3 × 4
##   cafe_mac     minutes_to_campus fave_temp hangout      
##   <chr>                    <dbl>     <dbl> <chr>        
## 1 Cheesecake                  15        18 the mountains
## 2 Cheese pizza                10        24 a beach      
## 3 udon noodles                 4        18 the mountains

# Last 6 rows (the tail) of the survey data set
tail(survey)
## # A tibble: 6 × 4
##   cafe_mac        minutes_to_campus fave_temp hangout 
##   <chr>                       <dbl>     <dbl> <chr>   
## 1 Burger                         10        21 a forest
## 2 Pepperoni Pizza                10        24 a beach 
## 3 Hamburger                       6        23 a beach 
## 4 Ginger Cookies                 10        26 a beach 
## 5 bbq chicken                     5        14 a city  
## 6 Breakfast food                 15        25 a city

# Names of the variables in the survey data set
names(survey)
## [1] "cafe_mac"          "minutes_to_campus" "fave_temp"        
## [4] "hangout"

# Structure of all variables in the survey data set
str(survey)
## spc_tbl_ [98 × 4] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
##  $ cafe_mac         : chr [1:98] "Cheesecake" "Cheese pizza" "udon noodles" "egg rolls" ...
##  $ minutes_to_campus: num [1:98] 15 10 4 7 5 35 5 15 7 20 ...
##  $ fave_temp        : num [1:98] 18 24 18 10 18 7 75 24 13 16 ...
##  $ hangout          : chr [1:98] "the mountains" "a beach" "the mountains" "a beach" ...
##  - attr(*, "spec")=
##   .. cols(
##   ..   cafe_mac = col_character(),
##   ..   minutes_to_campus = col_double(),
##   ..   fave_temp = col_double(),
##   ..   hangout = col_character()
##   .. )
##  - attr(*, "problems")=<externalptr>

Exercise 6: Code = communication

# Make it less smooshy. Add spaces!
seq(from = 1, to = 9, by = 2)
## [1] 1 3 5 7 9

# Use consistent spacing
seq(from = 1, to = 9, by = 2)
## [1] 1 3 5 7 9

# Use more descriptive names when storing objects
my_output <- -13

# Use a shorter and easier to read name
celsius_today <- -13
CelsiusToday  <- -13

Exercise 7: You will make so many mistakes!

Exercise 8: Your turn

# a
peaks <- read_csv("https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-09-22/peaks.csv")

# b
names(peaks)
## [1] "peak_id"                    "peak_name"                 
## [3] "peak_alternative_name"      "height_metres"             
## [5] "climbing_status"            "first_ascent_year"         
## [7] "first_ascent_country"       "first_ascent_expedition_id"

# c
dim(peaks)
## [1] 468   8
nrow(peaks)
## [1] 468

# d
head(peaks)
## # A tibble: 6 × 8
##   peak_id peak_name     peak_alternative_name height_metres climbing_status
##   <chr>   <chr>         <chr>                         <dbl> <chr>          
## 1 AMAD    Ama Dablam    Amai Dablang                   6814 Climbed        
## 2 AMPG    Amphu Gyabjen <NA>                           5630 Climbed        
## 3 ANN1    Annapurna I   <NA>                           8091 Climbed        
## 4 ANN2    Annapurna II  <NA>                           7937 Climbed        
## 5 ANN3    Annapurna III <NA>                           7555 Climbed        
## 6 ANN4    Annapurna IV  <NA>                           7525 Climbed        
## # ℹ 3 more variables: first_ascent_year <dbl>, first_ascent_country <chr>,
## #   first_ascent_expedition_id <chr>