Random samples, concept of a statistic and its distribution, sample mean as a measure of location and sample variance as a measure of spread.
Concept of likelihood; examples of likelihood for simple distributions. Estimation for a single unknown parameter by maximising likelihood. Examples drawn from: Bernoulli, binomial, geometric, Poisson, exponential (parametrized by mean), normal (mean only, variance known). Data to include simple surveys, opinion polls, archaeological studies, etc. Properties of estimators---unbiasedness, Mean Squared Error = (bias
\(^{2}\) + variance). Statement of Central Limit Theorem (excluding proof). Confidence intervals using CLT. Simple straight line fit,
\(Y_{t}=a+bx_{t}+\varepsilon_{t}\), with
\(\varepsilon _{t}\) normal independent errors of zero mean and common known variance. Estimators for
\(a\),
\(b\) by maximising likelihood using partial differentiation, unbiasedness and calculation of variance as linear sums of
\(Y_{t}\). (No confidence intervals). Examples (use scatter plots to show suitability of linear regression).
Linear regression with 2 regressors. Special case of quadratic regression
\(Y_t = a + bx_t + cx^2_t + \epsilon_t\). Model diagnostics and outlier detection. Residual plots. Heteroscedasticity.
Outliers and studentized residuals. High-leverage points and leverage statistics. [2.5]
Introduction to unsupervised learning with real world examples. Principal components analysis (PCA). Proof that PCs maximize directions of maximum variance and are orthogonal using Lagrange multipliers. PCA as eigendecomposition of covariance matrix. Eigenvalues as variances. Choosing number of PCs. The multivariate normal distribution pdf. Examples of PCA on multivariate normal data and clustered data. [3]
Clustering techniques; K-means clustering. Minimization of within-cluster variance. K-means algorithm and proof that it will decrease objective function. Local versus global optima and use of random initializations. Hierarchical clustering techniques. Agglomerative clustering using complete, average and single linkage [2.5]
