$$ \newcommand{\mybold}[1]{\boldsymbol{#1}} \newcommand{\trans}{\intercal} \newcommand{\norm}[1]{\left\Vert#1\right\Vert} \newcommand{\abs}[1]{\left|#1\right|} \newcommand{\bbr}{\mathbb{R}} \newcommand{\bbz}{\mathbb{Z}} \newcommand{\bbc}{\mathbb{C}} \newcommand{\gauss}[1]{\mathcal{N}\left(#1\right)} \newcommand{\chisq}[1]{\mathcal{\chi}^2_{#1}} \newcommand{\studentt}[1]{\mathrm{StudentT}_{#1}} \newcommand{\fdist}[2]{\mathrm{FDist}_{#1,#2}} \newcommand{\iid}{\overset{\mathrm{IID}}{\sim}} \newcommand{\argmin}[1]{\underset{#1}{\mathrm{argmin}}\,} \newcommand{\projop}[1]{\underset{#1}{\mathrm{Proj}}\,} \newcommand{\proj}[1]{\underset{#1}{\mybold{P}}} \newcommand{\expect}[1]{\mathbb{E}\left[#1\right]} \newcommand{\prob}[1]{\mathbb{P}\left(#1\right)} \newcommand{\dens}[1]{\mathit{p}\left(#1\right)} \newcommand{\var}[1]{\mathrm{Var}\left(#1\right)} \newcommand{\cov}[1]{\mathrm{Cov}\left(#1\right)} \newcommand{\sumn}{\sum_{n=1}^N} \newcommand{\meann}{\frac{1}{N} \sumn} \newcommand{\cltn}{\frac{1}{\sqrt{N}} \sumn} \newcommand{\trace}[1]{\mathrm{trace}\left(#1\right)} \newcommand{\diag}[1]{\mathrm{Diag}\left(#1\right)} \newcommand{\grad}[2]{\nabla_{#1} \left. #2 \right.} \newcommand{\gradat}[3]{\nabla_{#1} \left. #2 \right|_{#3}} \newcommand{\fracat}[3]{\left. \frac{#1}{#2} \right|_{#3}} \newcommand{\W}{\mybold{W}} \newcommand{\w}{w} \newcommand{\wbar}{\bar{w}} \newcommand{\wv}{\mybold{w}} \newcommand{\X}{\mybold{X}} \newcommand{\x}{x} \newcommand{\xbar}{\bar{x}} \newcommand{\xv}{\mybold{x}} \newcommand{\Xcov}{\mybold{M}_{\X}} \newcommand{\Xcovhat}{\hat{\mybold{M}}_{\X}} \newcommand{\Covsand}{\Sigmam_{\mathrm{sand}}} \newcommand{\Covsandhat}{\hat{\Sigmam}_{\mathrm{sand}}} \newcommand{\Z}{\mybold{Z}} \newcommand{\z}{z} \newcommand{\zv}{\mybold{z}} \newcommand{\zbar}{\bar{z}} \newcommand{\Y}{\mybold{Y}} \newcommand{\Yhat}{\hat{\Y}} \newcommand{\y}{y} \newcommand{\yv}{\mybold{y}} \newcommand{\yhat}{\hat{\y}} \newcommand{\ybar}{\bar{y}} \newcommand{\res}{\varepsilon} \newcommand{\resv}{\mybold{\res}} \newcommand{\resvhat}{\hat{\mybold{\res}}} \newcommand{\reshat}{\hat{\res}} \newcommand{\betav}{\mybold{\beta}} \newcommand{\betavhat}{\hat{\betav}} \newcommand{\betahat}{\hat{\beta}} \newcommand{\betastar}{{\beta^{*}}} \newcommand{\betavstar}{{\betav^{*}}} \newcommand{\loss}{\mathscr{L}} \newcommand{\losshat}{\hat{\loss}} \newcommand{\f}{f} \newcommand{\fhat}{\hat{f}} \newcommand{\bv}{\mybold{\b}} \newcommand{\bvhat}{\hat{\bv}} \newcommand{\alphav}{\mybold{\alpha}} \newcommand{\alphavhat}{\hat{\av}} \newcommand{\alphahat}{\hat{\alpha}} \newcommand{\omegav}{\mybold{\omega}} \newcommand{\gv}{\mybold{\gamma}} \newcommand{\gvhat}{\hat{\gv}} \newcommand{\ghat}{\hat{\gamma}} \newcommand{\hv}{\mybold{\h}} \newcommand{\hvhat}{\hat{\hv}} \newcommand{\hhat}{\hat{\h}} \newcommand{\gammav}{\mybold{\gamma}} \newcommand{\gammavhat}{\hat{\gammav}} \newcommand{\gammahat}{\hat{\gamma}} \newcommand{\new}{\mathrm{new}} \newcommand{\zerov}{\mybold{0}} \newcommand{\onev}{\mybold{1}} \newcommand{\id}{\mybold{I}} \newcommand{\sigmahat}{\hat{\sigma}} \newcommand{\etav}{\mybold{\eta}} \newcommand{\muv}{\mybold{\mu}} \newcommand{\Sigmam}{\mybold{\Sigma}} \newcommand{\rdom}[1]{\mathbb{R}^{#1}} \newcommand{\RV}[1]{{#1}} \def\A{\mybold{A}} \def\A{\mybold{A}} \def\av{\mybold{a}} \def\a{a} \def\B{\mybold{B}} \def\b{b} \def\S{\mybold{S}} \def\sv{\mybold{s}} \def\s{s} \def\R{\mybold{R}} \def\rv{\mybold{r}} \def\r{r} \def\V{\mybold{V}} \def\vv{\mybold{v}} \def\v{v} \def\vhat{\hat{v}} \def\U{\mybold{U}} \def\uv{\mybold{u}} \def\u{u} \def\W{\mybold{W}} \def\wv{\mybold{w}} \def\w{w} \def\tv{\mybold{t}} \def\t{t} \def\Sc{\mathcal{S}} \def\ev{\mybold{e}} \def\Lammat{\mybold{\Lambda}} \def\Q{\mybold{Q}} \def\eps{\varepsilon} $$

STAT151A Quiz 3

Please write your full name and email address here:

\[\\[2in]\]

Also, please put your intials on each page in case the pages get separated.

\[\\[1in]\]

You have 30 minutes for this quiz.

There are three questions, each weighted equally..

There are extra pages at the end if you need more space for solutions.

Question 1

Let $\y_n$, $\x_n$, and $\z_n$ all denote continuous, scalar–valued variables for $n=1,\ldots,N$.

Define

\[ a_n = \frac{\x_n + \z_n}{2} \quad\textrm{and}\quad d_n = \frac{\x_n - \z_n}{2}. \]

Let

$\Yhat_{1ad}$ denote the fit from the regression $\y_n \sim \gamma_0 + \gamma_a \a_n + \gamma_b d_n$, and
$\Yhat_{1xz}$ denote the fit from the regression $\y_n \sim \beta_0 + \beta_x \x_n + \beta_z \z_n$.

Please say whether each of the following statements is true, false, or cannot be determined from the information given.

(a) $\norm{\Yhat_{1ad}}^2 = \norm{\Yhat_{1xz}}^2$.

(b) $\betahat_0 + \betahat_x \x_n + \betahat_z \z_n = \gammahat_0 + \gammahat_a \a_n + \gammahat_b d_n$ for all $n$.

(c) $\norm{\Y - \Yhat_{1ad}}^2 < \norm{\Y - \Yhat_{1xz}}^2$.

(d) All the answers above follow from a single linear algebra fact about the relationship between $(1, \x_n, \z_n)^\trans$ and $(1, a_n, d_n)^\trans$. Briefly state this fact. A single sentence is enough.

Question 2

Recall the definitions:

\[ \begin{aligned} RSS :={}& \reshat^\trans \reshat & \textrm{(Residual sum of squares)}\\ TSS :={}& \Y^\trans \Y & \textrm{(Total sum of squares)}\\ ESS :={}& \Yhat^\trans \Yhat & \textrm{(Explained sum of squares)}\\ R^2 :={}& \frac{ESS}{TSS}. \end{aligned} \]

Consider two regressions, $\Y \sim \X \betav$ and $\Y \sim \X\gammav_z + \Z \gammav_z$. That is, the first regression is on $\X$ alone and the second regression is on both $\X$ and $\Z$. You may assume that all the columns of $\X$ and $\Z$ are mutually linearly independent so that $(\X, \Z)$, the matrix formed by taking the columns of $\X$ concatenated horizontally with the columns of $\Z$, is full column rank. Further, assume that $\Y$ is not orthogonal to any of the columns of $(\X, \Z)$.

Let $RSS_{X}$, $TSS_{X}$, $ESS_{X}$, and $R^2_{X}$ denote the correponding quantities from $\Y \sim \X$, and $RSS_{XZ}$, $TSS_{XZ}$, $ESS_{XZ}$, and $R^2_{XZ}$ the corresponding quantities from the regression $\Y \sim \X + \Z$. For example,

\[ ESS_{X} = (\X \betavhat)^\trans (\X \betavhat) \quad\textrm{and}\quad ESS_{XZ} = (\X \gammavhat_X + \Z \gammavhat_Z)^\trans (\X \gammavhat_X + \Z \gammavhat_Z), \] and so on.

Please say whether each of the following statements, true, false, or cannot be determined from the information given.

(a) $RSS_X > RSS_{XZ}$

(b) $ESS_X > ESS_{XZ}$

(c) $R^2_X > R^2_{XZ}$

(d) All the answers above follow from a single linear algebra fact about the column spans of $\X$ and $(\X, \Z)$. Briefly state this fact. A single sentence is enough.

Question 3

Suppose you have $N$ observations of the following data about an aircraft:

\[ \begin{aligned} \y_n ={}& \textrm{Drag force (strictly positive)} \\ \x_n ={}& \textrm{Wind speed (strictly positive)} \\ \z_n ={}& \textrm{Material type } \in \{0, 1 \}. \end{aligned} \]

Here, $\z_n = 1$ means that a new experimental material was used, and $\z_n = 0$ means the new material was not used. The regressor $\z_n$ is either $0$ or $1$.

You want to test the effect of the experimental material on the drag force taking into account variability in wind speed. Basic physics suggests that the drag force is proportional to $\x_n^2$.

(a)

Suppose you want to estimate the quantity $\betav = (\beta_1, \beta_z, \beta_x)^\trans$ in the regression

\[ \y_n = \beta_1 \beta_z^{\z_n} \x_n^{\beta_x} \exp(\res_n), \]

where $\res_n$ accounts for the error between $\y_n$ and $\beta_1 \beta_z^{\z_n} \x_n^{\beta_x}$. How can you transform the data in order to write the problem of estimating $\betav$ as a linear regression problem?

(b)

Suppose you define $\ell_n := \log \y_n$ run the regression \[ \ell_n \sim{} \gamma_1 + \gamma_z \z_n + \gamma_x \log \x_n \quad\textrm{so that}\quad \hat{\ell}_n ={} \gammahat_1 + \gammahat_z \z_n + \gammahat_x \log \x_n. \]

Define $f(\x_n, \z_n) = \exp(\hat{\ell}_n)$, and write an expression the value of $f(\x_n, 1) / f(\x_n, 0)$. You may give your answer in terms of the entries of $\gammav = (\gamma_1, \gamma_z, \gamma_x)^\trans$, $\x_n$, and $\z_n$ (though you may not need to use all of them).

(c)

Suppose we assume that \[ f(\x_n, \z_n) = \tilde{\beta}_1 \tilde{\beta}_z^{\z_n} \x_n^{\tilde{\beta}_x} \]

holds for all $\z_n$ and $\x_n$. Using this, write $\tilde{\beta}_1$, $\tilde{\beta}_x$, and $\tilde{\beta}_z$. in terms of $\gammahat_1$, $\gammahat_z$, and $\gammahat_x$.

(d)

Suppose we estimate that $f(\x_n, 1) / f(\x_n, 0) < 1$. If we take our estimate to be roughly correct, what does this imply for the effect of our experimental material on the drag?

Extra space for answers (indicate clearly which problem you are working on)