Template

Learn Statistics

Learn statistics: probability, distributions, estimation and inference, regression and experimental design, well enough to analyse data myself and to judge whether somebody else's analysis holds up.

This plan develops your statistical capability from mathematical and conceptual foundations through to experimental design and critical evaluation of published studies. Following your theory-first preference, each phase introduces foundational mathematical mechanics before applying them to empirical datasets using Python (SciPy, Statsmodels, Pandas) or R. At 8 hours per week the tasks in this 11-phase curriculum come to roughly five to seven months of rigorous study. Upon completion, you will be able to design experiments, derive estimators, build and diagnose generalised linear models from raw data, and systematically dissect external statistical claims for methodological vulnerabilities.

By the end: You will be able to formulate rigorous probabilistic models, execute parametric and non-parametric inference pipelines on raw datasets in Python, and write structured methodological critique reports identifying threats to statistical validity in published research.

Starting levelBeginnerStyleTheory first
8h / week10 phases42 tasks~185h total

This is the map — make this roadmap yours

It shows what this journey generally looks like. Tell Kaidoro your version of the goal and it builds the plan around where you are actually starting, what to do first, the hours you really have, and what you have already finished.

1

Exploratory Data Analysis and Descriptive Summaries

Establishes core descriptive metrics, data hygiene, and the foundational Python computing environment. This phase forms the baseline for all subsequent empirical tasks.

  • Set up Python statistical workspace with Jupyter and core libraries
    ~2hLearn

    A reproducible computing environment prevents software configuration issues from distracting you during mathematical and applied exercises.

    You'll learn

    • Virtual environments — isolated runtime environments for Python package management
    • Pandas DataFrame — tabular data structure supporting column-wise operations and indexing
    • NumPy ndarray — contiguous memory array enabling vectorised mathematical computations

    Configure an isolated Python scientific environment containing Pandas, NumPy, SciPy, Statsmodels, and Matplotlib. Verify your setup by loading a standard dataset and inspecting summary properties.

    Done when: your environment runs an end-to-end sanity script that imports all required libraries, loads a CSV, and executes basic array transformations without warnings or version conflicts.

    How to work through it

    1. Create a dedicated virtual environment using venv or Conda
    2. Install numpy, pandas, scipy, statsmodels, matplotlib, and jupyter
    3. Run a test Jupyter Notebook verifying imports and version compatibility
    4. Load the classic Iris or Palmer Penguins dataset into a Pandas DataFrame
  • Calculate and contrast measures of central tendency and dispersion
    ~3.5hLearn

    Summarising distributions requires understanding which metrics resist outlier distortion and which fail under asymmetry.

    You'll learn

    • Bessel's correction — using n-1 degrees of freedom to correct negative bias in sample variance
    • Breakdown point — the proportion of incorrect observations an estimator can handle before yielding arbitrary results
    • Interquartile range (IQR) — difference between 75th and 25th percentiles measuring spread robustly

    Study arithmetic mean, median, trimmed mean, sample variance, standard deviation, and interquartile range (IQR). Understand the mathematical properties of each statistic under skewness and contamination by heavy outliers.

    Done when: you have written a Python script computing these measures from first principles without relying on third-party statistical helper functions and verified the results against NumPy and SciPy.

    How to work through it

    1. Derive the mathematical definitions of sample variance with Bessel's correction (n-1)
    2. Implement custom Python functions for mean, median, variance, and IQR
    3. Simulate contaminated normal distributions with extreme outliers to observe breakdown points
    4. Verify the custom outputs against scipy.stats functions
  • Analyse skewness, kurtosis, and multi-modal distributions
    ~4hPractice

    Standard parametric procedures assume specific distributional shapes; mastering moments prevents misapplying these tools later.

    You'll learn

    • Skewness — measure of asymmetry in probability distribution around its mean
    • Excess kurtosis — measure of tail fatness and outlier propensity relative to a normal distribution
    • Empirical CDF (ECDF) — step function showing the proportion of observations less than or equal to x

    Examine higher-order sample moments including Fisher-Pearson skewness and excess kurtosis. Explore graphical techniques such as kernel density estimates, empirical cumulative distribution functions (ECDFs), and box plots to identify distribution geometry.

    Done when: you can inspect an unfamiliar tabular dataset and categorise each continuous column by symmetry, tail weight, and modality backed by both summary metrics and graphical diagnostics.

    How to work through it

    1. Formulate the equations for third and fourth standardised sample moments
    2. Compute skewness and kurtosis across synthetic exponential, normal, and bimodal data
    3. Plot histograms overlaid with kernel density estimates (KDE) and ECDFs
    4. Write a summary function that flags extreme skewness (> |1.0|) or heavy tails
  • Produce an exploratory data analysis report on an uncleaned empirical dataset
    ~5hBuild

    Synthesises descriptive metrics into a structured investigative workflow that serves as the entry gate for probabilistic modelling.

    You'll learn

    • Data imputation — replacing missing data with substituted values using defined rules
    • Correlation matrix — square table showing pairwise correlation coefficients between variables

    Select a real-world messy dataset (e.g., housing prices or census extracts). Clean anomalies, compute robust summaries, visualise bivariate associations, and document distributional properties.

    Done when: you produce an exploratory data analysis notebook detailing data cleaning decisions, robust univariate summaries, correlation matrices, and identified distributional characteristics.

    How to work through it

    1. Acquire an uncurated dataset with missing values and non-standard entries
    2. Identify and treat missingness and erroneous outliers with explicit rationale
    3. Generate a diagnostic panel combining robust central tendency, IQR, skewness, and ECDFs
    4. Document primary associations and anomalies in a coherent summary
2

Probability Fundamentals and Set Theory

Covers formal probability axioms, sample spaces, combinatorics, conditional probability, and Bayes' theorem. Can be studied concurrently with basic exploratory analysis.

  • Master Kolmogorov axioms, sample spaces, and set operations
    ~4hLearn1 resource

    Rigorous probability avoids the paradoxes and errors common in intuitive reasoning about chance.

    You'll learn

    • Kolmogorov axioms — three foundational statements defining mathematical probability measures
    • Sample space — set of all possible outcomes of a random experiment
    • Combinatorics — mathematical study of counting, arrangement, and combination

    Study sample spaces, event algebras, unions, intersections, complements, and the three Kolmogorov axioms. Prove basic properties of probability measures such as the addition rule and monotonicity.

    Done when: you can solve 10 formal set-theoretic probability problems without reference to answer keys, proving intermediate properties algebraically.

    How to work through it

    1. Define sample spaces and events as sets and subsets
    2. Work through formal proofs of basic probability propositions derived from axioms
    3. Apply De Morgan's laws to complex compound events
    4. Solve combinatoric counting problems involving permutations and combinations
  • Analyse conditional probability, independence, and the law of total probability
    ~4.5hPractice

    Statistical inference and graphical models depend directly on conditional independence structures.

    You'll learn

    • Stochastic independence — condition where the occurrence of one event does not alter the probability of another
    • Law of Total Probability — rule relating marginal probabilities to conditional probabilities across a partition
    • Conditional independence — independence of two events given the occurrence of a third

    Formulate conditional probability P(A|B), marginalisation, stochastic independence, and conditional independence. Work through counter-intuitive scenarios where marginal independence does not imply conditional independence.

    Done when: you can systematically decompose joint event probabilities into conditional factors and evaluate independence claims analytically and via simulation.

    How to work through it

    1. Derive the definition of conditional probability and the multiplication rule
    2. Formulate the Law of Total Probability over mutually exclusive partitions
    3. Simulate conditional probability puzzles (such as the Monty Hall problem) in Python to verify analytical solutions
    4. Prove whether pairs and triplets of events are mutually or conditionally independent
  • Apply Bayes' theorem to sequential belief updating
    ~4hBuild

    Bayes' theorem is the primary mathematical mechanism for updating knowledge in the presence of uncertainty.

    You'll learn

    • Bayes' Theorem — mathematical formula for determining conditional probability given prior knowledge and evidence
    • Base rate fallacy — cognitive error of ignoring general background rates in favour of specific evidence
    • Likelihood ratio — ratio of the probability of observing data under one hypothesis versus another

    Derive Bayes' Theorem from the definition of conditional probability. Study base rate fallacies, likelihood ratios, prior probabilities, and posterior distributions in diagnostic testing scenarios.

    Done when: you have written a Python simulator that performs sequential Bayesian updating across incoming diagnostic test results and plots belief trajectories.

    How to work through it

    1. Derive Bayes' theorem in both simple and partitioned forms
    2. Calculate posterior disease probability under varying base rates and test sensitivities/specificities
    3. Implement an updating function that ingests a sequence of binary test results and calculates step-by-step posteriors
    4. Visualise how low base rates dampen even highly specific diagnostic tests
3

Discrete and Continuous Random Variables

Builds the theory of random variables, probability mass/density functions, cumulative distribution functions, expectation, variance, and standard parametric families.

  • Formalise random variables, CDFs, PMFs, and PDFs
    ~4.5hLearn

    Random variables convert abstract event spaces into numerical functions suitable for mathematical analysis.

    You'll learn

    • Probability Density Function (PDF) — function whose integral over an interval gives probability
    • Cumulative Distribution Function (CDF) — function giving P(X <= x) for any random variable
    • Quantile function — generalised inverse of the CDF mapping probabilities back to values

    Study the mapping from sample spaces to real numbers. Understand the mathematical distinction between probability mass functions (PMF) for discrete variables and probability density functions (PDF) for continuous variables, alongside the universal properties of cumulative distribution functions (CDF).

    Done when: you can correctly calculate probabilities, quantiles, and densities by integrating PDFs and summing PMFs, explaining why P(X = x) = 0 for continuous variables.

    How to work through it

    1. Define discrete and continuous random variables formally
    2. Verify the mathematical conditions for valid PMFs and PDFs (non-negativity, integration/sum to 1)
    3. Derive the Cumulative Distribution Function (CDF) for given density functions
    4. Calculate quantiles and percentiles using the inverse CDF (quantile function)
  • Derive expectation, variance, and moment-generating functions
    ~5hLearn

    Expectation and variance are the foundational summary operators used across all estimation and inference.

    You'll learn

    • Linearity of expectation — property that E[aX + bY] = aE[X] + bE[Y] regardless of independence
    • Moment Generating Function (MGF) — alternative specification of a probability distribution providing a systematic path to moments
    • Covariance — measure of the joint variability of two random variables

    Define mathematical expectation E[X], variance Var(X), and covariance. Master the linearity of expectation and study Moment Generating Functions (MGFs) as tools for proving distribution properties and deriving moments.

    Done when: you have analytically derived the expectation and variance of linear combinations of random variables and computed moments using MGF derivatives.

    How to work through it

    1. Prove the linearity of expectation for dependent and independent variables
    2. Derive Var(aX + bY) under non-zero covariance
    3. Define Moment Generating Functions M_X(t) = E[e^(tX)]
    4. Derive the first two moments of standard variables by differentiating their MGFs at t=0
  • Master standard discrete distributions (Bernoulli, Binomial, Poisson, Geometric)
    ~4hPractice

    Standard discrete distributions model counts, occurrences, and binary outcomes in experimental and operational settings.

    You'll learn

    • Poisson distribution — discrete probability distribution expressing the probability of a given number of events occurring in a fixed interval
    • Binomial distribution — discrete distribution of the number of successes in n independent Bernoulli trials
    • Geometric distribution — distribution of the number of Bernoulli trials needed to get one success

    Examine discrete probability models. Derive their PMFs, expectations, and variances. Study real-world generation processes: Bernoulli trials, binomial counts, geometric waiting times, and Poisson count processes over continuous intervals.

    Done when: you can identify the correct discrete model for an unfamiliar real-world scenario, calculate exact probabilities analytically, and simulate the generative process in Python.

    How to work through it

    1. Derive Binomial PMF from combinatorics and Bernoulli trials
    2. Derive Poisson distribution as the limiting case of the Binomial distribution (Law of Rare Events)
    3. Implement Python simulations for Binomial and Poisson arrival processes
    4. Fit Poisson and Binomial models to count data and assess goodness of fit
  • Master standard continuous distributions (Uniform, Normal, Exponential, Gamma)
    ~4.5hPractice

    Continuous models form the analytical substrate for error distributions, survival times, and asymptotic statistical theory.

    You'll learn

    • Normal (Gaussian) distribution — symmetric continuous distribution characterised by mean and variance
    • Exponential distribution — probability distribution describing the time between events in a Poisson point process
    • Memoryless property — property where the future probability distribution depends only on current state, not elapsed time

    Explore fundamental continuous distributions. Derive the memoryless property of the Exponential distribution, study the scaling and location parameters of the Normal distribution, and examine how Gamma and Beta distributions generalise continuous modelling.

    Done when: you can transform continuous random variables, prove the memoryless property of the Exponential distribution, and compute standardised normal probabilities (z-scores) from scratch.

    How to work through it

    1. Derive the standardisation formula Z = (X - mu) / sigma for the Gaussian distribution
    2. Prove analytically that the Exponential distribution is memoryless: P(X > s + t | X > s) = P(X > t)
    3. Plot and compare family shapes across varying scale and shape parameters using SciPy
    4. Compute probability intervals and tail densities for skewed continuous variables
  • Build a parametric distribution fitting and diagnostic tool in Python
    ~5hBuild

    Solidifies your understanding of distribution theory by forcing you to evaluate how closely mathematical idealisations match messy empirical data.

    You'll learn

    • Quantile-Quantile (Q-Q) plot — visual diagnostic tool comparing empirical quantiles against theoretical distribution quantiles
    • Log-likelihood — logarithm of the likelihood function used to measure model fit quality

    Develop a structured Python module that accepts an empirical dataset, fits candidate discrete or continuous distributions using SciPy, overlays fitted densities on empirical ECDFs, and generates Q-Q (quantile-quantile) plots.

    Done when: your tool ingests raw real-world data, evaluates at least three candidate distribution families, and outputs diagnostic Q-Q plots alongside log-likelihood goodness-of-fit scores.

    How to work through it

    1. Implement theoretical quantile calculations for normal, exponential, and uniform distributions
    2. Construct quantile-quantile (Q-Q) plotting routines from raw arrays without high-level library shortcuts
    3. Fit candidate parameters via maximum likelihood helpers in scipy.stats
    4. Package the routines into an executable Python script with clear diagnostic visualisations
4

Joint Distributions, Limit Theorems, and Sampling Theory

Investigates joint, marginal, and conditional probability distributions, followed by the Law of Large Numbers and the Central Limit Theorem.

  • Compute joint, marginal, and conditional distributions
    ~4.5hLearn

    Multivariate analysis and regression modelling depend directly on conditional expectations and joint probability structures.

    You'll learn

    • Joint probability density — function describing the simultaneous distribution of two or more random variables
    • Law of Total Variance — decomposition of variance into unexplained variation and explained variation
    • Marginal distribution — probability distribution of a subset of collection of random variables without reference to values of the other variables

    Formulate bivariate joint densities f(x, y) and joint PMFs. Calculate marginal distributions via integration/summation, compute conditional expectations E[Y|X], and derive the law of total variance (Eve's law).

    Done when: you can solve bivariate integration problems to obtain marginals, conditional densities, and covariance for both discrete and continuous cases.

    How to work through it

    1. Integrate joint continuous densities over specified 2D support regions to find marginals
    2. Calculate conditional expectations E[Y|X = x] and conditional variance Var(Y|X = x)
    3. Prove and apply the Law of Total Variance: Var(Y) = E[Var(Y|X)] + Var(E[Y|X])
    4. Compute the correlation coefficient rho and explain why rho = 0 does not imply independence
  • Simulate and prove the Law of Large Numbers and Central Limit Theorem
    ~4hPractice

    The Central Limit Theorem provides the theoretical justification for using normal approximations in inference without knowing the exact parent population distribution.

    You'll learn

    • Weak Law of Large Numbers (WLLN) — sample average converges in probability to the expected value as n grows
    • Central Limit Theorem (CLT) — standardised sample mean of i.i.d. random variables converges in distribution to standard normal
    • Standard Error (SE) — standard deviation of the sampling distribution of a statistic (sigma / sqrt(n))

    Study convergence in probability (Weak Law of Large Numbers) and convergence in distribution (Central Limit Theorem). Use Monte Carlo simulations to observe sample means converging to normality regardless of the underlying parent distribution's shape.

    Done when: you have constructed interactive simulations illustrating the CLT across highly skewed, uniform, and bimodal parent distributions with varying sample sizes (n = 2 to n = 1000).

    How to work through it

    1. State the formal mathematical conditions and bounds for LLN and CLT (finite mean and variance)
    2. Write a Monte Carlo script sampling from uniform, Cauchy, and Pareto distributions
    3. Observe convergence failures in heavy-tailed distributions lacking finite variance (e.g., Cauchy)
    4. Visualise standard error decay at the theoretical rate of 1 / sqrt(n)
  • Derive sampling distributions (Chi-Square, Student's t, F-distribution)
    ~4.5hLearn

    Exact small-sample inference relies on these exact sampling distributions rather than asymptotic normal approximations.

    You'll learn

    • Chi-Square distribution — continuous distribution of the sum of the squares of k independent standard normal random variables
    • Student's t-distribution — symmetric continuous distribution with heavier tails than the normal, used when sample variance is estimated
    • F-distribution — continuous distribution arising from the ratio of two independent chi-square variables divided by their degrees of freedom

    Study distributions that arise from sampling normal populations: the Chi-Square distribution (sum of squared standard normals), Student's t-distribution (ratio of normal to square root of chi-square), and the F-distribution (ratio of two independent chi-squares).

    Done when: you can state the exact derivation of t, Chi-Square, and F distributions from standard normal components and explain when each distribution governs a test statistic.

    How to work through it

    1. Derive the Chi-Square distribution with k degrees of freedom as the sum of k squared standard normal variables
    2. Construct Student's t distribution as the ratio Z / sqrt(V / nu) and inspect heavy tails for small degrees of freedom
    3. Define the F distribution as the ratio of two scaled Chi-Square variables
    4. Plot probability densities showing convergence of Student's t to Standard Normal as degrees of freedom approach infinity
  • Produce a sampling distribution simulation report
    ~3.5hBuild

    Solidifies the theoretical bridge between single sample observations and long-run sampling distributions.

    You'll learn

    • Monte Carlo simulation — computational algorithm relying on repeated random sampling to obtain numerical results
    • Sampling distribution — probability distribution of a given statistic based on a random sample

    Write a self-contained computational notebook demonstrating the behaviour of sample estimators (mean, median, variance) under repeated sampling from diverse populations, verifying theoretical standard errors empirically.

    Done when: your notebook shows empirical sampling distributions matching theoretical predictions for means and variances across varying sample sizes, complete with quantified Monte Carlo error margins.

    How to work through it

    1. Simulate 10,000 independent samples of varying size n from skewed Gamma distributions
    2. Compute sample mean and sample variance across all repetitions
    3. Compare empirical standard error of the mean against theoretical sigma / sqrt(n)
    4. Plot overlay curves of theoretical t and normal densities against empirical sampling histograms
5

Estimation Theory and Confidence Intervals

Explores point estimation, properties of estimators (bias, consistency, efficiency), Maximum Likelihood Estimation (MLE), and exact/bootstrap confidence intervals.

  • Evaluate estimator properties: Bias, Variance, MSE, and Consistency
    ~4.5hLearn

    Inference requires principled criteria to select the best estimator among multiple competing formulas.

    You'll learn

    • Bias of an estimator — difference between an estimator's expected value and the true value of the parameter
    • Mean Squared Error (MSE) — expected value of the squared differences between the estimator and the parameter
    • Consistency — property that as sample size increases, the estimator converges in probability to the true parameter

    Define point estimators. Calculate estimator bias Bias(theta_hat) = E[theta_hat] - theta, variance, and Mean Squared Error (MSE = Bias^2 + Variance). Formulate consistency as n approaches infinity.

    Done when: you can mathematically prove whether an estimator is unbiased and consistent, and demonstrate the bias-variance trade-off on competing estimators.

    How to work through it

    1. Prove that the sample mean is an unbiased and consistent estimator of the population mean
    2. Demonstrate why maximum likelihood variance (dividing by n) is biased, while dividing by n-1 is unbiased
    3. Calculate the MSE of competing estimators to identify which achieves lower overall risk
    4. Simulate estimator paths across growing sample sizes to show consistency graphically
  • Derive Maximum Likelihood Estimators (MLE) and Fisher Information
    ~5.5hLearn

    Maximum Likelihood Estimation is the foundational parameter estimation engine used across modern statistics and machine learning.

    You'll learn

    • Maximum Likelihood Estimation (MLE) — method of estimating parameters by maximising the likelihood function
    • Fisher Information — way of measuring the amount of information that an observable random variable carries about an unknown parameter
    • Cramér-Rao Lower Bound — theorem stating the minimum possible variance for any unbiased estimator

    Formulate likelihood functions L(theta|x) and log-likelihood functions. Derive analytical MLEs for Bernoulli, Normal, and Poisson parameters. Compute Fisher Information and the Cramér-Rao Lower Bound to evaluate minimum variance unbiased estimators.

    Done when: you can set up log-likelihood functions, take first and second derivatives to locate maxima, and calculate the asymptotic variance of estimators via the Fisher Information matrix.

    How to work through it

    1. Formulate joint likelihood functions for i.i.d. observations
    2. Differentiate log-likelihood equations and solve score equations for parameter estimates
    3. Compute the second derivative (Hessian) to confirm concavity and calculate Fisher Information
    4. Implement numerical MLE optimisation in Python using scipy.optimize.minimize for intractable models
  • Construct exact and asymptotic Confidence Intervals
    ~4hPractice

    Confidence intervals convey parameter uncertainty with rigorous long-run coverage guarantees.

    You'll learn

    • Confidence Interval (CI) — range of values that would contain the true parameter in a specified percentage of repeated samples
    • Pivotal quantity — function of sample data and parameters whose probability distribution does not depend on unknown parameters
    • Wilson score interval — robust asymmetric confidence interval for binomial proportions overcoming Wald interval flaws

    Formulate confidence intervals via pivotal quantities. Construct exact z-intervals and t-intervals for population means, Wilson score intervals for proportions, and Chi-Square intervals for variance. Master the exact frequentist interpretation of confidence intervals.

    Done when: you can calculate exact and asymptotic confidence intervals for means, differences in means, and proportions, and correctly state the frequentist coverage definition without committing common misinterpretations.

    How to work through it

    1. Derive pivotal quantities for normal mean with known and unknown population variance
    2. Construct 95% and 99% Student's t confidence intervals for continuous sample means
    3. Compute Wilson score and Wald confidence intervals for binomial proportions
    4. Write a simulation demonstrating that 95% of generated intervals actually capture the true parameter across repeated runs
  • Implement non-parametric and parametric Bootstrap intervals
    ~4.5hBuild

    Bootstrapping enables accurate estimation of confidence intervals without imposing strict parametric distributional assumptions.

    You'll learn

    • Resampling with replacement — sampling method where an entity once selected is returned to the pool before the next draw
    • Percentile bootstrap — constructing confidence intervals directly from empirical quantiles of bootstrap replications
    • Plug-in principle — estimating population functionals by evaluating the same functional on the empirical distribution

    Study the empirical bootstrap principle. Implement resampling with replacement to construct empirical bootstrap distributions, standard bootstrap intervals, and percentile bootstrap confidence intervals for statistics lacking analytical formulas (e.g., medians, ratio of variances).

    Done when: you have written a Python bootstrap pipeline that generates percentile and BCa (bias-corrected accelerated) intervals for non-standard summary statistics on empirical data.

    How to work through it

    1. Explain the core bootstrap analogy: sample is to population as resample is to sample
    2. Implement a vectorized bootstrap resampling loop in NumPy
    3. Compute bootstrap standard errors and percentile confidence intervals for sample medians and trimmed means
    4. Compare bootstrap intervals against theoretical t-intervals on skewed datasets
6

Classical Hypothesis Testing and Invariance

Covers the Neyman-Pearson and Fisherian hypothesis testing frameworks, Type I and II errors, power, p-values, t-tests, ANOVA, and non-parametric alternatives.

  • Deconstruct Null Hypothesis Significance Testing (NHST)
    ~4hLearn1 resource

    Statistical testing errors stem almost entirely from conflating Fisherian evidence with Neyman-Pearson decision rules.

    You'll learn

    • Type I error — rejection of a true null hypothesis (false positive)
    • Type II error — failure to reject a false null hypothesis (false negative)
    • Statistical power — probability of correctly rejecting a false null hypothesis (1 - beta)

    Contrast Fisher's inductive p-value philosophy with the Neyman-Pearson decision-theoretic framework. Define null (H0) and alternative (H1) hypotheses, significance level alpha, Type I error rate (false positive), Type II error rate beta (false negative), and statistical power (1 - beta).

    Done when: you can articulate the formal definition of a p-value without fallacies (e.g., proving H0 is false), map rejection regions, and compute Type I and Type II error trade-offs.

    How to work through it

    1. Formulate null and alternative hypotheses as complementary parameter subspaces
    2. Define the formal definition of a p-value: P(Data as or more extreme | H0 is true)
    3. Map alpha levels to critical values in standard sampling distributions
    4. Calculate Type II error rate beta for a specified true effect size and fixed sample size
  • Execute one-sample, two-sample, and paired Student's and Welch's t-tests
    ~4.5hPractice

    Comparing group averages is the most ubiquitous statistical inference task across experimental disciplines.

    You'll learn

    • Welch's t-test — two-sample location test used to test the hypothesis that two populations have equal means without assuming equal variance
    • Cohen's d — standardised effect size measuring the distance between two means in units of standard deviation
    • Homoscedasticity — assumption that different groups have equal or similar variances

    Derive test statistics for comparing means. Compare the standard two-sample Student's t-test (assuming equal variance) with Welch's t-test (unpooled variance, adjusted Satterthwaite degrees of freedom). Implement paired t-tests for repeated measures.

    Done when: you can verify test assumptions (normality, homoscedasticity), execute the appropriate t-test variant using SciPy, and interpret effect size (Cohen's d) alongside p-values and confidence intervals.

    How to work through it

    1. Derive the test statistic for Welch's t-test and calculate adjusted degrees of freedom
    2. Check distributional assumptions using Shapiro-Wilk test and Levene's test for equality of variance
    3. Execute paired and unpaired t-tests on empirical experiment datasets
    4. Calculate Cohen's d point estimates and confidence intervals for effect size
  • Conduct One-Way and Two-Way Analysis of Variance (ANOVA)
    ~5hPractice

    Running multiple pairwise t-tests causes severe alpha inflation; ANOVA provides principled simultaneous group comparisons.

    You'll learn

    • Analysis of Variance (ANOVA) — collection of statistical models used to analyse differences among group means
    • Family-Wise Error Rate (FWER) — probability of making one or more Type I errors when performing multiple hypotheses tests
    • Tukey's HSD test — single-step multiple comparison procedure to find means that are significantly different from each other

    Decompose total sum of squares into between-group and within-group components. Derive the F-statistic for testing equality of multiple group means. Perform post-hoc pairwise multiple comparison corrections (Tukey's HSD, Bonferroni) to control Family-Wise Error Rate (FWER).

    Done when: you can build an ANOVA table from raw group data, interpret main effects and interaction terms in Two-Way ANOVA, and compute post-hoc tests with corrected alpha levels.

    How to work through it

    1. Derive the Sum of Squares Total (SST), Treatment (SSB), and Error (SSW) decompositions
    2. Construct an ANOVA table computing Mean Squares and the F-ratio statistic
    3. Fit One-Way and Two-Way ANOVA models using statsmodels.formula.api
    4. Apply Tukey's Honestly Significant Difference (HSD) test to control family-wise error
  • Apply non-parametric tests (Mann-Whitney U, Wilcoxon Signed-Rank, Kruskal-Wallis)
    ~4hPractice

    Empirical data frequently violates normality; non-parametric rank tests provide robust inference without distributional transformation.

    You'll learn

    • Non-parametric statistics — statistical methods that do not rely on parameterised probability distribution families
    • Mann-Whitney U test — non-parametric test of the null hypothesis that two populations have identical distributions
    • Kruskal-Wallis test — rank-based non-parametric test to determine if there are differences between two or more groups

    Study rank-based inference when normality assumptions fail or when working with ordinal scales. Implement the Mann-Whitney U test (two independent samples), Wilcoxon signed-rank test (paired samples), and Kruskal-Wallis test (k independent samples).

    Done when: you can decide when to switch from parametric to rank-based tests, perform the calculations in Python, and interpret rank sums and median shift hypotheses correctly.

    How to work through it

    1. Explain the rank transformation mechanism and its invariance to monotonic transformations
    2. Execute Mann-Whitney U test and interpret the stochastic dominance hypothesis
    3. Run Wilcoxon signed-rank test on paired non-normal differences
    4. Execute Kruskal-Wallis test as the non-parametric analogue to One-Way ANOVA
  • Build a comprehensive hypothesis testing pipeline and automated diagnostic report
    ~5hBuild

    Forces you to operationalise the entire decision tree of classical hypothesis testing into an explicit, reproducible framework.

    You'll learn

    • Statistical pipeline — programmatic sequence of analytical steps transforming raw data into final inference results
    • Assumption checking — testing whether data satisfies prerequisite conditions required by inferential models

    Write a Python script that takes two or more numerical groups, automatically tests normality and homoscedasticity assumptions, routes the data to the appropriate test (Student's t, Welch's t, ANOVA, or Mann-Whitney/Kruskal-Wallis), and returns effect sizes and confidence intervals.

    Done when: your automated pipeline accepts an arbitrary multi-group dataset and outputs a clean statistical decision report specifying tested assumptions, selected test justification, test statistics, exact p-values, and standardised effect sizes.

    How to work through it

    1. Implement automated assumption testing for normality (Shapiro-Wilk) and variance homogeneity (Levene)
    2. Implement conditional branching selecting Welch's t, Student's t, or Mann-Whitney U based on diagnostic results
    3. Calculate effect size point estimates and confidence intervals for all outputs
    4. Validate the automated script on three diverse benchmark datasets with known properties
7

Linear Regression Modelling and Diagnostics

Develops the theory of Ordinary Least Squares (OLS) regression, geometric projections, Gauss-Markov assumptions, residual diagnostics, and multiple regression.

  • Derive Ordinary Least Squares (OLS) estimators geometrically and algebraically
    ~5hLearn

    Understanding the matrix algebra and geometric projection of OLS clarifies all linear and generalised linear models.

    You'll learn

    • Ordinary Least Squares (OLS) — method for estimating unknown parameters in a linear regression model by minimising squared residuals
    • Projection matrix (Hat matrix) — orthogonal projection matrix mapping the response vector onto the fitted value space
    • Normal equations — system of linear equations whose solution yields OLS parameter estimates

    Formulate simple and multiple linear regression models Y = X*beta + epsilon. Derive the closed-form normal equations beta_hat = (X^T * X)^(-1) * X^T * Y using matrix calculus and geometric orthogonal projection of Y onto the column space of X.

    Done when: you have derived the normal equations on paper from first principles and written a Python function implementing matrix inversion to estimate regression parameters without high-level library abstractions.

    How to work through it

    1. Formulate the sum of squared residuals objective function in matrix notation
    2. Take the vector derivative with respect to beta and set to zero to derive normal equations
    3. Interpret the hat matrix H = X(X^T X)^(-1) X^T as an orthogonal projection operator
    4. Implement custom OLS estimation in NumPy computing beta, fitted values, and residuals
  • Evaluate the Gauss-Markov theorem and standard inference in OLS
    ~4.5hLearn

    Inference on regression coefficients is valid only when underlying Gauss-Markov assumptions hold.

    You'll learn

    • Gauss-Markov Theorem — states that OLS estimator is BLUE (Best Linear Unbiased Estimator) under classical conditions
    • Exogeneity — assumption that independent variables are uncorrelated with the model error term
    • Coefficient of Determination (R^2) — proportion of variance in the dependent variable explained by independent variables

    Examine the Gauss-Markov assumptions: linearity, exogeneity E[epsilon|X]=0, homoscedasticity Var(epsilon|X)=sigma^2*I, and no multicollinearity. Prove that OLS is the Best Linear Unbiased Estimator (BLUE). Derive standard errors and t-tests for coefficients.

    Done when: you can list each Gauss-Markov condition, explain the consequences if any condition is violated, and calculate parameter standard errors and R-squared manually.

    How to work through it

    1. State and formalise the five classical Gauss-Markov assumptions
    2. Prove that beta_hat is an unbiased estimator of beta under exogeneity
    3. Derive the variance-covariance matrix of the parameter estimates: Var(beta_hat) = sigma^2 * (X^T X)^(-1)
    4. Derive R-squared and Adjusted R-squared measures of variance explained
  • Perform comprehensive residual diagnostics and assumption validation
    ~4.5hPractice

    Unchecked regression models frequently yield false significant relationships due to residual patterning and influential outliers.

    You'll learn

    • Heteroscedasticity — circumstance in which the variability of a variable is unequal across the range of values of a second variable
    • Cook's distance — measure of the influence of a single observation when performing a least-squares regression analysis
    • Variance Inflation Factor (VIF) — measure of the amount of multicollinearity in a set of multiple regression variables

    Conduct deep residual analysis to detect violations of regression assumptions. Evaluate heteroscedasticity (Breusch-Pagan test), non-normality of residuals (Q-Q plots, Jarque-Bera), autocorrelation (Durbin-Watson), and influential outliers (Cook's distance, leverage points).

    Done when: you can inspect residual plots from an OLS fit, run formal diagnostic tests in Statsmodels, and identify specific assumption violations and high-leverage data points.

    How to work through it

    1. Plot residuals against fitted values to detect non-linear patterns and heteroscedastic fan shapes
    2. Perform the Breusch-Pagan test for heteroscedasticity and apply White's robust standard errors (HC3)
    3. Calculate leverage values (hat values) and Cook's distance to isolate influential observations
    4. Assess multicollinearity using the Variance Inflation Factor (VIF)
  • Model multiple regression with interactions and categorical covariates
    ~4.5hPractice

    Real-world causal systems involve moderating variables and categorical factors that require interaction modelling.

    You'll learn

    • Interaction effect — situation where the simultaneous effect of two variables on a third is not additive
    • Dummy variable — numerical variable used in regression analysis to represent subgroups of the sample in your study
    • Wald test — parametric statistical test to confirm whether parameters are simultaneously equal to zero

    Construct multiple regression models incorporating dummy variables for categorical predictors, continuous-by-categorical interaction terms, and polynomial transformations. Interpret main effects in the presence of interactions.

    Done when: you can fit a multiple regression model with interaction terms using Statsmodels, correctly interpret all marginal effects and interaction coefficients in writing, and test joint hypotheses using F-tests.

    How to work through it

    1. Implement one-hot dummy coding with explicit reference categories
    2. Specify and fit interaction models (Y ~ X1 + X2 + X1:X2) in Statsmodels
    3. Calculate and interpret marginal effects of X1 across distinct values of moderator X2
    4. Execute general linear hypothesis tests (Wald F-tests) for multiple parameter subsets simultaneously
  • Produce an end-to-end linear regression empirical report
    ~5hBuild

    Demonstrates mastery in translating raw data into a fully diagnosed, defensible linear regression model.

    You'll learn

    • Akaike Information Criterion (AIC) — estimator of prediction error and relative quality of statistical models for a given dataset
    • Robust standard errors (Huber-White) — standard error estimates that remain consistent under heteroscedasticity

    Select an observational dataset with multiple continuous and categorical predictors. Fit a baseline multiple OLS model, run full diagnostic checks, correct for heteroscedasticity using robust covariance estimators, and present a structured analysis report with model comparison metrics (AIC, BIC, Adjusted R^2).

    Done when: you produce a complete regression report detailing the problem statement, baseline OLS model, formal diagnostic tests, remediation steps, and clear coefficient interpretations with confidence intervals.

    How to work through it

    1. Select an empirical multi-feature dataset (e.g., wage data or real estate metrics)
    2. Fit the initial OLS model specification and generate standard summary tables
    3. Execute residual diagnostics (heteroscedasticity, VIF, influence) and document issues
    4. Refit using robust standard errors or non-linear terms, and compare models via AIC/BIC
8

Categorical Data Analysis and Generalized Linear Models

Expands regression beyond normal errors to categorical and count responses using Generalized Linear Models (GLMs), focusing on Logistic and Poisson regression.

  • Analyse 2x2 contingency tables and categorical associations
    ~4hLearn

    Many empirical phenomena involve counts and binary states where continuous normal models cannot apply.

    You'll learn

    • Odds ratio — statistic that quantifies the strength of the association between two events
    • Pearson's Chi-Square test — test of independence between two categorical variables
    • Fisher's exact test — statistical significance test used in the analysis of contingency tables valid for all sample sizes

    Study categorical data analysis: joint frequencies, marginal probabilities, risk differences, relative risk, and odds ratios. Conduct Pearson's Chi-Square Test of Independence and Fisher's Exact Test for small sample frequencies.

    Done when: you can compute odds ratios and confidence intervals by hand, perform Chi-Square tests of independence in Python, and explain when to use Fisher's exact test.

    How to work through it

    1. Construct 2x2 and r x c contingency tables from raw data
    2. Calculate relative risk, odds ratios, and log-odds with asymptotic standard errors
    3. Compute Pearson's Chi-Square test statistic and expected cell frequencies
    4. Apply Fisher's exact test using hypergeometric probabilities for small sample cells
  • Master Generalized Linear Models (GLM) theoretical structure
    ~4.5hLearn1 resource

    Understanding the general GLM framework enables you to model non-normal data (binary, count, proportions, survival) without learning fragmented ad-hoc techniques.

    You'll learn

    • Link function — function providing the relationship between the linear predictor and the mean of the distribution function
    • Exponential dispersion family — set of probability distributions sharing a specific mathematical structure including Normal, Binomial, and Poisson
    • Iteratively Reweighted Least Squares (IRLS) — optimisation algorithm used to find maximum likelihood estimates for GLMs

    Study the unified GLM framework: the exponential dispersion family, the linear predictor eta = X*beta, and the monotonic link function g(mu) = eta. Understand how OLS, Logistic, and Poisson regressions are special cases within this single unifying architecture.

    Done when: you can state the three components of a GLM (random component, systematic component, link function) and write down the likelihood equations for canonical link models.

    How to work through it

    1. Define the exponential dispersion family density function f(y; theta, phi)
    2. Express Bernoulli and Poisson distributions in canonical exponential family form
    3. Derive canonical link functions (identity, logit, log) for standard distributions
    4. Explain the Iteratively Reweighted Least Squares (IRLS) algorithm used to fit GLMs
  • Fit and interpret Logistic Regression models
    ~5hPractice

    Logistic regression is the standard method for modelling binary decisions, risks, and classification problems with valid probabilities.

    You'll learn

    • Logit function — quantile function associated with the standard logistic distribution (log-odds)
    • Odds Ratio (Logistic) — factor by which the odds of an outcome change for a one-unit increase in a predictor
    • Receiver Operating Characteristic (ROC) — graphical plot showing the diagnostic ability of a binary classifier system across discrimination thresholds

    Formulate binary logistic regression using the logit link function log(p / (1-p)) = X*beta. Derive log-likelihood functions, estimate coefficients using Statsmodels GLM / Logit, and transform coefficients into exponentiated Odds Ratios. Evaluate model calibration and discrimination.

    Done when: you can fit a multiple logistic regression model, interpret odds ratios and their confidence intervals, and assess model performance using ROC curves, AUC, and the Hosmer-Lemeshow calibration test.

    How to work through it

    1. Derive the logistic log-likelihood function and verify its convexity
    2. Fit logistic models using statsmodels.formula.api.logit
    3. Convert raw logit coefficients into Odds Ratios via exponentiation (exp(beta)) and interpret them
    4. Plot Receiver Operating Characteristic (ROC) curves, calculate Area Under Curve (AUC), and evaluate Brier scores
  • Model count data with Poisson and Negative Binomial regression
    ~4.5hPractice

    Real count data almost universally violates the Poisson equidispersion assumption; mastering Negative Binomial regression prevents severely deflated standard errors.

    You'll learn

    • Equidispersion — property of the Poisson distribution where the variance equals the mean
    • Overdispersion — presence of greater variability in a dataset than would be expected based on a given statistical model
    • Negative Binomial regression — generalised linear model for count data that handles overdispersion by introducing a gamma-distributed dispersion parameter

    Formulate Poisson regression using the log link function. Address the strict equidispersion assumption (Mean = Variance). Identify and model overdispersion using Quasi-Poisson and Negative Binomial regression models.

    Done when: you can test for overdispersion in a Poisson regression fit and fit a Negative Binomial model when overdispersion is present, correctly interpreting incidence rate ratios.

    How to work through it

    1. Fit baseline Poisson regression models for event count data using log link
    2. Compute the dispersion parameter phi = Pearson Chi-Square / degrees of freedom to diagnose overdispersion
    3. Fit Negative Binomial regression models to account for unobserved heterogeneity
    4. Interpret exponentiated coefficients as Incidence Rate Ratios (IRR)
  • Build a categorical and count data empirical analysis notebook
    ~4.5hBuild

    Consolidates GLM theory into a finished empirical project matching professional analytical workflows.

    You'll learn

    • Model Deviance — measure of goodness of fit for GLMs comparing the model to a saturated model
    • Incidence Rate Ratio (IRR) — relative measure of the effect of a given factor on the rate of occurrence of an event

    Analyse an observational dataset containing binary or count outcomes (e.g., customer churn, clinical diagnoses, hospital admissions). Fit logistic or negative binomial GLMs, perform deviance goodness-of-fit tests, and document the findings.

    Done when: you have produced an executed notebook demonstrating data exploration, model fitting with link function justification, overdispersion or calibration checks, and substantive interpretation of exponentiated parameters.

    How to work through it

    1. Select an open dataset with a binary or discrete count dependent variable
    2. Formulate the GLM specification and fit model parameters using statsmodels
    3. Perform goodness-of-fit diagnostics (deviance, residuals, calibration)
    4. Generate a parameter interpretation table presenting odds ratios or rate ratios with 95% confidence intervals
9

Experimental Design, Power Analysis, and Causal Principles

Covers principles of rigorous experimentation: randomisation, blocking, statistical power calculation, sample size determination, and basic causal inference.

  • Master principles of experimental design: Randomisation, Blocking, and Control
    ~4hLearn1 resource

    No amount of clever statistical modelling can rescue data generated by a fundamentally flawed experimental design.

    You'll learn

    • Randomisation — random allocation of experimental units to treatment groups to balance unobserved confounders
    • Blocking — arranging experimental units into homogeneous groups (blocks) to reduce residual error variance
    • Confounding variable — unmeasured variable that influences both the supposed cause and the supposed effect

    Study Fisher's three principles of experimental design: randomisation (controlling unobserved confounders), replication (estimating experimental error), and local control/blocking (reducing unexplained variance). Study Completely Randomized Designs (CRD) and Randomized Complete Block Designs (RCBD).

    Done when: you can design an experimental allocation matrix given specific nuisance variables, write blocking structures, and explain how randomisation protects against confounding.

    How to work through it

    1. Explain the mathematical difference between random sampling and random assignment
    2. Construct a Randomized Complete Block Design (RCBD) balancing known covariates across treatments
    3. Derive the expected mean squares for treatment and block factors in ANOVA models
    4. Explain the role of placebo controls, blinding, and counterbalancing in eliminating experimental bias
  • Perform analytical and simulation-based Power Analysis and Sample Size Sizing
    ~4.5hPractice

    Underpowered experiments produce high false negative rates and inflate the effect sizes of discovered true positives (winner's curse).

    You'll learn

    • Power analysis — calculation used to determine the minimum sample size required to detect an effect of a given size with specified confidence
    • Winner's Curse (Type M error) — exaggeration of estimated effect size in studies that achieve statistical significance with low power
    • Type S error — error of getting the sign of an effect wrong in an underpowered study

    Formulate statistical power calculations relating alpha, sample size n, effect size (Cohen's d or delta), and power (1 - beta). Perform analytical power sizing for t-tests and proportions, and build Monte Carlo simulation routines to size complex or non-standard models.

    Done when: you can determine the required sample size for a planned experiment analytically and build a custom simulation function to estimate power for models without closed-form power equations.

    How to work through it

    1. Calculate required sample size for two-sample t-tests and proportion tests using statsmodels.stats.power
    2. Construct power curves visualising power across varying sample sizes and effect sizes
    3. Write a Monte Carlo power simulator that generates synthetic datasets under an alternative hypothesis to compute empirical rejection rates
    4. Explain the Winner's Curse (Type M and Type S errors) in underpowered studies
  • Explore Causal Inference fundamentals: Potential Outcomes and DAGs
    ~4.5hLearn1 resource

    Distinguishing true causal effects from observational correlations requires formal causal calculus.

    You'll learn

    • Potential Outcomes Framework — Neyman-Rubin model defining causality in terms of hypothetical counterfactual states
    • Directed Acyclic Graph (DAG) — structural graphical model representing causal paths between variables
    • Collider bias — distortion that arises when conditioning on a common effect of two variables

    Study the Rubin Causal Model (Potential Outcomes framework Y(0), Y(1)) and Pearl's Directed Acyclic Graphs (DAGs). Define Average Treatment Effect (ATE), selection bias, confounding, and colliders. Learn the backdoor criterion for selecting conditioning variables.

    Done when: you can draw a DAG for a causal system, identify confounders versus colliders, and explain why conditioning on a collider introduces spurious associations.

    How to work through it

    1. Define potential outcomes Y(1) and Y(0) and the Fundamental Problem of Causal Inference
    2. Express Average Treatment Effect (ATE) and Average Treatment Effect on the Treated (ATT)
    3. Construct Directed Acyclic Graphs (DAGs) using networkx or DAGitty to map causal mechanisms
    4. Identify backdoor paths and apply the backdoor criterion to select necessary adjustment sets
  • Produce an experimental design and analysis plan (Pre-Analysis Protocol)
    ~4.5hBuild

    Pre-specification prevents researcher degrees of freedom, p-hacking, and post-hoc rationalisation in empirical investigations.

    You'll learn

    • Pre-analysis plan — document specifying the exact hypotheses, data collection procedures, and statistical models prior to looking at data
    • Minimum Detectable Effect (MDE) — smallest true effect size that an experiment has a specified probability of detecting

    Develop a formal experimental protocol and power analysis for a hypothetical or real experiment (e.g., A/B test, clinical trial, or operational intervention). Include randomisation scheme, sample size justification, primary/secondary metrics, and planned statistical model specification.

    Done when: you produce a complete, structured pre-analysis document containing sample size calculations, power curves, randomisation code, and model specifications ready for implementation.

    How to work through it

    1. Formulate primary null and alternative hypotheses with minimum detectable effect size (MDE)
    2. Execute power analysis justifying planned sample size across baseline conversion/variance estimates
    3. Write the Python script implementing block randomisation and balance verification
    4. Draft the complete pre-specified statistical analysis model, including covariates and outlier protocols
10

Critical Statistical Review and Methodological Auditing

Develops the critical capability to dissect external analyses, detect methodological vulnerabilities, evaluate replication integrity, and identify p-hacking or publication bias.

  • Deconstruct statistical malpractices: P-hacking, HARKing, and Researcher Degrees of Freedom
    ~4hLearn1 resource

    Auditing external claims requires knowing the exact mechanisms researchers use (intentionally or accidentally) to fabricate statistical significance.

    You'll learn

    • P-hacking — practice of manipulating data analysis until non-significant results become significant
    • HARKing — presenting a post-hoc hypothesis as if it were an a priori hypothesis
    • Researcher degrees of freedom — flexibility in data collection, analysis, and reporting that enables obtaining statistically significant results

    Study systemic failure modes in empirical science: p-hacking (data dredging), HARKing (Hypothesizing After Results are Known), selective outcome reporting, optional stopping, and flexible covariate adjustment. Review Simmons et al.'s 'False-Positive Psychology'.

    Done when: you can simulate how flexible stopping rules and multiple subgroup testing artificially inflate false-positive rates from nominal 5% to over 60%.

    How to work through it

    1. Read Simmons et al. (2011) on researcher degrees of freedom
    2. Write a Python simulation running sequential t-tests with optional stopping after each observation to demonstrate alpha inflation
    3. Simulate multi-outcome testing without FWER correction to observe false discovery rates
    4. Document the red flags in methodology sections that signal post-hoc exploration disguised as confirmatory testing
  • Audit publication bias, p-curves, and replication diagnostics
    ~4.5hPractice

    Judging whether a body of evidence holds up requires evaluating whole literatures for selective reporting and publication filters.

    You'll learn

    • Funnel plot — scatter plot of treatment effect against a measure of study precision used to detect publication bias
    • P-curve — distribution of statistically significant p-values across independent studies used to evaluate evidential value
    • Publication bias — phenomenon where studies with positive results are significantly more likely to be published than null findings

    Examine the distribution of published findings across disciplines. Study funnel plots for detecting publication bias, p-curve analysis to distinguish true evidentiary value from p-hacking, and replication metrics.

    Done when: you can generate a funnel plot, perform an Egger's regression test for funnel plot asymmetry, and construct a p-curve to evaluate whether a literature holds true evidential value.

    How to work through it

    1. Construct synthetic meta-analytic effect sizes under selective publication filters
    2. Generate funnel plots and execute Egger's linear regression test for funnel asymmetry
    3. Extract reported p-values from a cluster of related papers and plot the empirical p-curve
    4. Interpret whether the p-curve exhibits right-skew (real effect), flatness (null effect), or left-skew (p-hacked null)
  • Perform a comprehensive statistical critique of an empirical research paper
    ~5.5hReview

    Synthesises your theoretical, computational, and critical knowledge into the ultimate target skill: independently judging whether someone else's analysis holds up.

    You'll learn

    • Statistical auditing — systematic review of data collection, analytical models, and inferential claims to assess empirical validity
    • Methodological reproducibility — ability to recreate the analytical findings from raw data and evaluate their robustness to alternative specifications

    Select an empirical research paper (from economics, medicine, social sciences, or industry data science) containing regression, hypothesis testing, or experimental claims. Systematically audit the methodology: verify whether model assumptions were tested, evaluate sample size power, check for potential confounders or collider bias, and assess robustness.

    Done when: you produce a formal 4-page written statistical critique evaluating the paper's data generation process, model validity, potential p-hacking risks, and whether the empirical evidence supports the claimed conclusions.

    How to work through it

    1. Select a published empirical paper with accessible methodology and reported statistical tests
    2. Extract reported sample sizes, effect sizes, test statistics, and baseline covariates
    3. Conduct a step-by-step audit against the statistical checklist: assumptions, power, multiple testing, GLM link validity, and confounding
    4. Author a structured critique report with explicit verdict on methodological soundness

How the plan fits together

10 phases in 9 stages. Anything on the same row can be worked on at the same time, and 2 of them can start straight away.

An arrow points from a phase to the work it unlocks: before starting any phase, every phase with an arrow into it has to be finished first.

STARTSTAGE 2STAGE 3STAGE 4STAGE 5STAGE 6STAGE 7STAGE 8STAGE 91Exploratory Data Analysisand Descriptive Summaries4 tasks · ~15h2Probability Fundamentalsand Set Theory3 tasks · ~13h3Discrete and ContinuousRandom Variables5 tasks · ~23h4Joint Distributions, LimitTheorems, and SamplingTheory4 tasks · ~17h5Estimation Theory andConfidence Intervals4 tasks · ~19h6Classical HypothesisTesting and Invariance5 tasks · ~23h7Linear RegressionModelling and Diagnostics5 tasks · ~24h8Categorical Data Analysisand Generalized LinearModels5 tasks · ~23h9Experimental Design, PowerAnalysis, and CausalPrinciples4 tasks · ~18h10Critical StatisticalReview and MethodologicalAuditing3 tasks · ~14h

Resources

13 in this plan's library, beyond the links on individual tasks.

Core Theory Texts & Courseware

Primary textbooks covering mathematical statistics, probability, and linear models.

  • All of Statistics: A Concise Course in Statistical Inference

    Essential concise treatment bridging parametric inference, maximum likelihood, asymptotic theory, and non-parametrics.

    Larry Wasserman (Springer) · Book · ~£60 · Advanced

  • An Introduction to Statistical Learning (ISLP / ISLR)

    Use for applied treatment of ordinary least squares, Gauss-Markov assumptions, residual diagnostics, and logistic regression.

    statlearning.com · Springer · Book · Free digital / paid print · Intermediate

  • Applied Linear Statistical Models (Fifth Edition)

    The definitive comprehensive reference for linear regression mechanics, matrix formulations, collinearity, and residual diagnostics.

    Michael H. Kutner, Christopher J. Nachtsheim, John Neter, and William Li (McGraw-Hill) · Book · ~£80 · Advanced

  • Causal Inference: What If

    Provides a rigorous foundation for distinguishing observational association from causation, target trials, and confounding control.

    hsph.harvard.edu · Miguel A. Hernán and James M. Robins (CRC Press) · Book · Free online PDF version · Advanced

  • Introduction to Probability (2nd Edition)

    Use to master intuitive story proofs, conditioning, standard parametric families, transformations, and Markov chains.

    probabilitybook.net · Chapman & Hall/CRC Press · Book · Free digital / paid print · Intermediate

  • OpenIntro Statistics (4th Edition)

    Use for foundational descriptive metrics, data collection hygiene, t-tests, ANOVA, and basic linear models.

    openintro.org · OpenIntro · Book · Free digital / paid print · Beginner

  • Statistical Rethinking: A Bayesian Course with Examples in R and Stan (2nd Edition)

    Use to build generative models, causal DAGs, and generalized linear mixed models from a Bayesian perspective.

    xcelab.net · Chapman & Hall/CRC Press · Book and Video Lectures · Free lectures and code / paid print book · Advanced

Computational Tools & Libraries

Python and R statistical computing documentation and interactive environments.

  • Statsmodels Documentation and API Reference

    Use as the primary Python computational toolkit for regression modeling, GLMs, maximum likelihood estimation, and statistical diagnostics.

    statsmodels.org · Statsmodels Development Team · Documentation · Free · Intermediate

  • statsmodels: Econometric and Statistical Modeling with Python

    The primary Python library for OLS, GLMs, statistical tests, and detailed regression diagnostic output.

    statsmodels.org · statsmodels developers · Official Documentation · Free · Intermediate

Methodological & Meta-Science Resources

Papers, p-curve guides, and checklists for critiquing statistical research.

  • False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant

    The foundational paper illustrating how researcher degrees of freedom and p-hacking artificially inflate false-positive rates.

    Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn (Psychological Science) · Research Paper · Free preprint / Open access via publisher · Intermediate

  • Improving Your Statistical Inferences

    Use to learn a priori power analysis, equivalence testing, likelihood ratios, and methodological auditing of empirical studies.

    lakens.github.io · Eindhoven University of Technology · Online Textbook and Courseware · Free textbook / audit free on Coursera · Intermediate

  • The Garden of Forking Paths: Why Multiple Comparisons Can Be a Problem, Even When There Is No 'Fishing Expedition' or 'p-Hacking'

    Crucial reading on subtle researcher flexibility where analyses are contingent on data without deliberate malfeasance.

    stat.columbia.edu · Andrew Gelman and Eric Loken (Columbia University) · Research Paper · Free · Intermediate

  • The Official User-Guide to the P-Curve and Evidential Value Resources

    Use to analyze p-value distributions across literature sets to detect p-hacking, publication bias, and evidential value.

    p-curve.com · Data Colada and p-curve.com · User Guide · Free · Intermediate