Introduction to Statistical Machine Learning

Notes and in-class exercises

You can download the .qmd file for this activity here and open in R-studio. The rendered version is posted in the course website (Activities tab). I often experiment with the class activities (and see it in live!) and make updates, but I always post the final version before class starts. To be sure you have the most up-to-date copy, please download it once you’ve settled in before class begins.

Welcome


Settling in

  • Sit in groups of 4 (preferably) or 3 (at least).
  • Share your names and pronouns (again!?). Discuss a high point of your holiday season.


  • Open the online manual:
    • https://mutasim221b.github.io/Mac-STAT-155-Fall-25/ (also linked in Moodle)
    • Top bar < Activities < Collecting and Summarizing Data
    • We will do some reading, live-note taking, in-class activities, solving examples, doing exercises…until we ran out of time!

Learning Goals

  • Identify the appropriate task (regression, classification, unsupervised) for a given research question
  • Develop foundation to be able to formulate research questions that align with regression, classification, or unsupervised learning tasks

Brief Overview of STAT 253

What’s Machine Learning?

“Machine Learning” was coined back in 1959 by Arthur Samuel, an early contributor to AI.

From Kohavi & Provost (1998): Machine Learning is the exploration & application of algorithms that can learn from existing patterns and make predictions using data.

IMPORTANT: humans are in charge of the “exploration & application”!

From James et al (2021) [link]: Statistical Learning refers to a vast set of tools for understanding data.


In STAT 253 we will…

  • Pick up where STAT 155 left off, acquiring tools that can be used to learn from data in greater depth and a wider variety of settings. (STAT 155 is a foundational subset of ML!)

  • Explore universal ML concepts using tools and software common among statisticians (hence “statistical” machine learning).

  • Survey a breadth of modern ML tools and algorithms that fall into the workflow below. Part of the cognitive load will be:

    • keeping all the tools in place (what are they and when to use them)

    • understanding the connections between the tools

    • adapting (not memorizing) code to implement each tool

    • a new topic almost every day

  • We’ll focus on concepts and applications over mathematical theory. (Come chat with me in office hours if you’re interested in learning more about the theory!)

Course Structure

Before Class

In order to dedicate our class time to hands-on learning, you will prepare for class by watching short videos, reading from our textbook, and completing short quizzes (checkpoints) to assess your initial understanding of concepts. You can reattempt each checkpoint question multiple times, with a small penalty for incorrect responses.

During Class

During class time, you will engage with each other in exercises and discussions that build upon the pre-class work. Please bring your laptop to class every day. Consistent attendance and active participation in these activities is expected of all students and, most importantly, will be crucial for your learning!

After Class

After class, you will be expected to finish any remaining exercises from the class activity and review/organize your notes. For each unit, you will also complete homework assignments designed to help you practice and synthesize material and provide an opportunity to receive feedback to further guide your learning.



Notes: Machine Learning Overview

Types of ML Tasks

Statistical machine learning tools can be classified as follows:

  • supervised or unsupervised
  • within supervised learning: regression vs classification
  • within unsupervised learning: clustering vs dimension reduction

Knowing which of these scenarios your research question falls into is an important first step in identifying which tool to use!


Supervised Learning

We want to model the relationship between some output variable1 \(y\) and input variables2 \(x = (x_1, x_2,..., x_p)\):

\[ \begin{split} y & = f(x) + \varepsilon \\ & = \text{(trend in the relationship) } + \text{ (residual deviation from the trend)} \\ \end{split} \]

Types of supervised learning tasks:

  • regression: \(y\) is quantitative
    example:
    \(y\) = number of dental caries (cavities)
    \(x\) = (genetic information at millions of markers, sex, age, age\(^2\), etc)
    project details

  • classification: \(y\) is categorical
    example:
    \(y\) = whether a patient experienced adverse surgery outcomes after undergoing an upper endoscopy (yes, no)
    \(x\) = (administration of sedation [anesthesia professional, nurse], age, medical comorbidities [eg sleep apnea], etc.)
    project details


Unsupervised Learning

We have some input variables \(x = (x_1, x_2,..., x_p)\) but there’s no output variable \(y\). Thus the goal is to use \(x\) to understand and/or modify the structure of our data.

Types of unsupervised learning tasks:

  • clustering: Identify and examine groups or clusters of data points that are similar with respect to their \(x_i\) values.
    example:
    \(x\) = (genetic data)
    project details (led by a Mac alum!)

  • dimension reduction: Turn the original set of \(p\) input variables, which are potentially correlated, into a smaller set of \(k < p\) variables which still preserve the majority of information in the originals.
    example:
    \(x\) = (genetic data)
    project details

Supplemental Figure 2A from Barragan et al (2023) [link] uses both clustering and dimension reduction!



Exercises

Instructions

  • Discuss the following scenarios as a group, talking through your ideas, questions, and reasoning as you go.
  • Write down your answers, and any insights or questions that come up while working, in your notebook or simply type in here and render your own work!
  • I’ll move around to groups to check in on your progress and see what questions you have.
  • You can check your answers by clicking the drop-down “Solutions” button.


Questions

Indicate whether each scenario below represents a regression, classification, or clustering task.

  1. How is the number of people that rent bikes on a given day in Washington, D.C. (\(y\)) explained by the temperature (\(x_1\)) and whether or not it’s a weekend (\(x_2\))?

Solution regression. there’s a quantitative output variable \(y\).



  1. Given the observed bill length (\(x_1\)) and bill depth (\(x_2\)) on a set of penguins, how many different penguin species might there be?

Solution clustering. there’s no output variable \(y\).



  1. How can we determine whether somebody has a certain infection (\(y\)) based on two different blood sample measurements, Measure A (\(x_1\)) and Measure B (\(x_2\))?

Solution classification. there’s a categorical output variable \(y\).



  1. Machine learn about past students! Scenario A.
    The following data were collected from past STAT 253 students and were analyzed using a machine learning algorithm. The questions were about their major (STAT/DS and Other), Activity (readng, streaming), walk time to MAC, Photo rating, class year, ). In your groups: (1) brainstorm what research question is being investigated; (2) determine whether this is a regression, classification, or clustering task; and (3) summarize what the output tells you about the students.

Solution
  1. predict someone’s major based on ‘other’ survey responses
  2. classification (\(y\) = major is categorical)
  3. (will vary by semester – what do you learn about the majors represented in this class and the variables that are useful for predicting it?)



  1. Machine learn about past students! Scenario B.
    Same directions as for Scenario A: (1) brainstorm what research question is being investigated; (2) determine whether this is a regression, classification, or clustering task; and (3) summarize what the output tells you about the students.

Solution
  1. predict walk time to Mac based on photo rating and class year
  2. regression (\(y\) = time to mac is quantitative)
  3. (answers will vary by semester – what do you learn about the relationships between these variables?)



  1. Use Spotify users’ previous listening behavior to identify groups of similar users.
Solution clustering



  1. Predict workers’ wages by their years of experience.
Solution regression (\(y\) = wages)



  1. Predict workers’ wages by their college major.
Solution regression (\(y\) = wages)



  1. Use a customer’s age to predict whether they’ve seen the Barbie movie.
Solution classification (\(y\) = whether or not watched the film)



  1. Look for similarities among genetic samples taken from a group of patients.
Solution clustering (no outcome \(y\))



What’s next?

What to work on after class today:

    • review the checkpoint instructions & policies on Moodle before you start!



Footnotes

  1. otherwise known as outcome, response, dependent variable↩︎

  2. otherwise known as predictors, features, independent variables↩︎