Probability

Core probability for research-facing work in CS, AI, and engineering.

Keywords

probability, conditioning, random variables, central limit theorem, concentration

1 Why This Module Matters

Probability is the language of uncertainty, repeated sampling, noisy data, and random systems.

Without it, a large part of statistics, ML, control, simulation, and learning theory turns into memorized formulas instead of understandable structure.

This module is the first pass through that structure. It starts from sample spaces and events, turns them into random variables and distributions, summarizes them by expectation and variance, then climbs toward asymptotic laws and finite-sample concentration.

Prerequisites Calculus, algebra, and basic proof comfort

Unlocks Statistics, learning theory, stochastic optimization, high-dimensional probability

Research Use Generalization, uncertainty quantification, randomized algorithms, stochastic models

2 First Pass Through This Module

1. Sample Spaces, Events, and Conditioning
2. Random Variables and Distributions
3. Expectation, Variance, Covariance
4. Joint, Conditional, and Bayes
5. Law of Large Numbers and CLT
6. Concentration and Common Inequalities

On a first pass, stay on the concept pages. You should not need future proof, lab, or research pages just to understand the core story of the module.

3 Navigation Rule

Use the module overview and core concept pages as the main first-pass route.

Each concept page should be strong enough to stand on its own for first-pass learning. Companion pages, when they exist, should feel optional rather than required.

4 Core Concepts

• Sample Spaces, Events, and Conditioning: builds the event-level language of probability and the idea of restricting the world after new information arrives.
• Random Variables and Distributions: turns outcomes into numerical quantities and introduces PMFs, PDFs, and CDFs.
• Expectation, Variance, Covariance: summarizes center, spread, and dependence.
• Joint, Conditional, and Bayes: explains how two random quantities interact and how evidence updates beliefs.
• Law of Large Numbers and CLT: separates stabilization of averages from the shape of their fluctuations.
• Concentration and Common Inequalities: gives explicit finite-sample tail control.

5 Proof Patterns In This Module

• Condition on the right event: many problems simplify once the correct information structure is made explicit.
• Use linearity before expanding: expectation and covariance calculations are often cleaner through identities than raw enumeration.
• Separate asymptotic from finite-sample reasoning: LLN and CLT are not the same kind of statement as Chebyshev or Hoeffding.

6 Applications

6.1 Statistical Inference

Probability explains what data variability means, how posteriors update after evidence, and why sample averages, standard errors, and uncertainty statements behave the way they do.

6.2 Learning and Randomized Computation

Empirical risk, stochastic gradients, randomized sketches, and generalization guarantees all lean on the same core ideas: conditioning, random variables, averages, and concentration.

7 Go Deeper By Topic

7.1 Joint, Conditional, and Bayes

Start with Joint, Conditional, and Bayes.

If you want one strong reinforcement path after the main page:

• revisit Sample Spaces, Events, and Conditioning
• then compute marginals, conditionals, and a Bayes update from one concrete table

7.2 Law of Large Numbers and CLT

Start with Law of Large Numbers and CLT.

If you want one strong next step after the main page:

• continue to Concentration and Common Inequalities

8 Optional Deep Dives After First Pass

Until dedicated companion pages land, the best deeper pass is through official course materials:

• MIT RES.6-012 lecture notes - watch for the progression from conditioning to random variables to inequalities. Checked 2026-04-24.
• Penn State STAT 414 - watch how the course separates foundational event language, distributions, bivariate structure, and limit laws. Checked 2026-04-24.

9 Study Order

The intended first pass is strictly the six concept pages above.

You are ready to move on when you can:

• model a problem with the right sample space or random variables
• compute basic summaries like expectation and variance
• interpret conditionals and Bayes updates
• explain the difference between LLN, CLT, and concentration

If one of those is still shaky, revisit the corresponding concept page before moving into statistics or learning-theory material.

10 Sources and Further Reading

• Harvard Stat 110 - First pass - strong official course hub with clear examples and excellent intuition. Checked 2026-04-24.
• Penn State STAT 414 - First pass - official open notes with a clean full-sequence treatment from foundations through CLT. Checked 2026-04-24.
• MIT RES.6-012 lecture notes - Second pass - official MIT notes with a theory-first bridge to inference and limit theorems. Checked 2026-04-24.

Sources checked online on 2026-04-24:

• Harvard Stat 110 course homepage
• Penn State STAT 414 overview
• MIT RES.6-012 lecture notes page