$$$$

STAT151A Code homework 4: Due March 8th

This coding assignment will use your work from homework 3 as a starting point. For the assignment, we’ll assume that

• $y_n={\mathit{\beta }}^{⊺}{\mathit{x}}_n+{\epsilon }_n$ for all $n$, including new datapoints
• ${\epsilon }_n\sim \mathcal{N}(0,{\sigma }^2)$
• The regressors ${\mathit{x}}_n$ are also random with covariance matrix ${\mathbf{\Sigma }}_{\mathit{X}}$.

1 Variability in the training set

Fix $N=500$, $P=3$, and set $\mathit{\beta }$ to some values you choose. Set ${\mathbf{\Sigma }}_{\mathit{X}}$ to have correlation $0.9$ off the diagonal and $1.0$ on the diagonal. Set ${\sigma }^2={\beta }^{⊺}{\mathbf{\Sigma }}_{\mathit{X}}\beta$.

Take ${\mathit{x}}_{\mathrm{new}}$ to be a single fixed draw from the distribution of regressors, and draw a large number (> 5000) of ${\epsilon }_{\mathrm{new},i}$, giving a large number of draws from $y_{\mathrm{new},i}|{\mathit{x}}_{\mathrm{new}}$. The $y_{\mathrm{new},i}$ should be normally distributed with mean ${\mathit{x}}_{\mathrm{new}}^{⊺}\beta$ and variance ${\sigma }^2$.

(a)

Draw a single training set $\mathit{X}$, $\mathit{Y}$, and use it to construct an 80% interval for $y_{\mathrm{new}}$. Find the proportion of $y_{\mathrm{new},i}$ that lie in the interval.

(b)

Repeat (a), but with 10 different training sets. You can keep the $y_{\mathrm{new},i}$ the same. For each different training set, plot the corresponding intervals. Are they different from one another?
By a lot or a little?

(c)

Repeat (b), but with $N=20$. You can keep the $y_{\mathrm{new},i}$ the same. How do the results compare to (b)? Why?

(d)

Repeat (b), but with $\sigma$ very small: specifically, set ${\sigma }^2=0.01{\beta }^{⊺}{\mathbf{\Sigma }}_{\mathit{X}}\beta$. You will need to draw new $y_{\mathrm{new},i}$. How do the results compare to (b)?

(e)

Repeat (b), but now take ${\mathit{x}}_{\mathrm{new}}$ to be the smallest eigenvector of ${\mathbf{\Sigma }}_{\mathit{X}}$. (You can find the smallest eigenvector of ${\mathbf{\Sigma }}_{\mathit{X}}$ using the R function eigen.)
You will need to draw new $y_{\mathrm{new},i}$. How do the results compare to (b)?