1. Variance decomposition

1.1 恒等式

Untitled

SSY = SSE + SSR ↓↓↓

$$ \sum_i(y_i-\bar{y})^2 = \sum_i (y_i-\hat{y}_i)^2 + \sum_i (\hat{y}_i-\bar{y})^2 $$

直观的动态图例请参考:https://online.stat.psu.edu/stat501/lesson/2/2.5

1.2 证明

对$y_i - \bar{y} = (y_i - \hat{y_i}) + (\hat{y_i} - \bar{y})$两边同时平方:

$$ (y_i - \bar{y})^2 = (y_i - \hat{y_i})^2 + (\hat{y_i} - \bar{y})^2+2(y_i - \hat{y_i})(\hat{y_i} - \bar{y}) $$

下面需要证明$\sum_{i=1}^n(y_i - \hat{y_i})(\hat{y_i} - \bar{y})=0$:

$$ \begin{equation*} \begin{aligned} \text{LHS} &= \sum_ie_i(\hat{y_i}-\bar{y}) \\ &= \sum_ie_i\hat{y_i}-\sum_ie_i\bar{y} \\ &= 0-0=0 \\ \end{aligned} \end{equation*} $$

Details: You need to use the results of OLS >>> OLS = taking partial derivatives with respect to α and β, then set to 0:

$$ \frac{\partial }{\partial \alpha} \sum_i (y_i-\hat{y}_i)^2 = 0 \ \text{→}\ \sum_i 2(y_i-\hat{y}_i)=0 \text{→}\ \sum_ie_i = 0\\

\frac{\partial }{\partial \beta} \sum_i (y_i-\hat{y}_i)^2 = 0\ \text{→}\ \sum_i 2(y_i-\hat{y}_i)x_i=0 \ \text{→}\ \sum_ie_ix_i = 0 $$

Then we can derive:

$$ \sum_i e_i \hat{y_i} = \sum_i e_i (\hat{\alpha} + \hat{\beta} x_i) = \hat{\alpha}\sum_i e_i + \hat{\beta} \sum_i e_ix_i = \hat{\alpha}·0 + \hat{\beta}·0=0 $$

1.3 Test of significance