• Least squares analysis of linear models.
• Gauss Markov Theorem.
• Estimability and testability of parameters.
• Confidence regions and prediction regions.
• Introduction to design of experiments.
• Analysis of variance.
• Factorial and block designs.
• Analysis of random, fixed and mixed models.
• Components of variance.
• Distribution of linear and quadratic forms in normal vectors.

Lecture Notes by Prof. M. Zhu

• STAT553-Notes(2014.08.24).docx

• A Note on Constrained Least Squares Estimation

R Code

• The sweep operator

Homework Assignments

R Source Code

Lecture Notes [updated on Feb. 06, 2012]

• Linear regression using auxiliary variables: a preview (Zhang et al., 2011).
  1. The univariate normal N(0, 1): a look at the simplest possible case (p=1, n=1) [How to make inference if a < mu < b ?]
  2. The bivariate normal: distribution, combining information, and prediction
  3. The bivariate normal (Cont'd): conditional distribution and the sweep operator

• The multivariate normal distribution
  1. Linear transformations
  2. Conditional distributions
  3. Simulation
  4. Quadratic forms
  5. Multivariate student-t
  6. F-distributions

• Estimation
  • Combining information, Sufficiency, and Dimension reduction
  • Unbiased estimators
    • The method of Least-Squares (LS): Is the idea deep?

      The Gauss-Markov theorem, named after Carl Friedrich Gauss and Andrey Markov, states that in a linear regression model in which the errors have expectation zero and are uncorrelated and have equal variances, the best linear unbiased estimator (BLUE) of the coefficients is given by the ordinary least squares (OLS) estimator.

    • The method of Maximum Likelihood (ML): Is the idea deep enough?
    • The distribution of sufficient statistics
    • How about the "median estimators" as a better alternative of "unbiased estimators"?
  • Distribution theory [and marginal inference]
  • Chi-square distribution
  • Student-t distribution
  • Bayesian inference and Robust inference; See also Bayesian Robust Multivariate Linear Regression with Incomplete Data
  • Techinical details in estimation with linear restrictions and design matrix of less than full rank
    • Method of LS/ML via Lagrange Multipliers
    • Method of combining information

• Hypothesis Testing
  • Difficulties in hypothesis testing for scientific inference: Fisher's significance testing, Rubin's posterior prodictive checking, the Neyman-Pearson decision theory, and statistical inference
  • Inference with the "toy example": X_i ~ N(mu_i, 1), i=1 and 2, with unknown mu = (mu_1, mu_2).
  • A general hypothesis for linear combinations of the regression coefficients is expessed in the form H: A β = c, where A and c are known.
    • Likelihood ratio test. What is the likelihood principle?
    • F-test. How to balance features of the alternative hypothesis?
  • Goodness-of-fit test.
  • Miscellaneous topics.

• Confidence Intevals and Regions
  • Parameters of interest and nuisance parameters
  • Marginalizing out nuisance parameters
  • Simultaneous inference. Why consider simultaneous confidence intervals (not mathematically but scientifically)?
  • Sheffe's S-method Is it optimal in some sense?
  • (Classifications of multiple assertions and) testing a set of assertions simultaneously
  • Prediction

• Analysis of Variance [simple structured models]
  • One-way classification: the problem of interest is that all the means are the same --- sufficient statistics and marginalization (inference about a point assertion/hypothesis on a (I-1) unknown quantity)
  • One-way classification: ``Empirical Bayes''
  • A problem with more than one unknown variance: the Behrens-Fisher problem
  • Two-way classification: the problem of interest is that there are no interactions --- sufficient statistics and marginalization (explained with the 2x2 case)

• Computational methods
  • Introduction to Matrix Computation (see Chapter 1)
  • The sweep operator in R
  • SVD in R

• Variable Selection
  • Variable Selection
  • Mallows' Cp (Mallows, 1995)
• The final-exam schedule: