Weighted Least Squares

• variance 가 input 마다 다른 것
  • $p(\mathbf{y}\mid \mathbf{x};\mathbf{\theta })=\mathcal{N}\left(\mathbf{y}\mid \mathbf{X}\mathbf{w},{\mathbf{\Lambda }}^{-1}\right)$ where $\mathbf{\Lambda }=\mathrm{diag}\left(1/{\sigma }^2\left({\mathbf{x}}_n\right)\right)$
• weighted least squares estimate
  • $\hat{\mathbf{w}}={\left({\mathbf{X}}^{\top }\mathbf{\Lambda }\mathbf{X}\right)}^{-1}{\mathbf{X}}^{\top }\mathbf{\Lambda }\mathbf{y}$ (MLE 를 사용)
• 예시) heteroskedastic regression
  • $p(y\mid \mathbf{x};\mathbf{\theta })=\mathcal{N}\left(y\mid {\mathbf{w}}^{\top }\mathbf{x},{\sigma }^2(\mathbf{x})\right)=\frac{1}{\sqrt{2\pi {\sigma }^2(\mathbf{x})}}\exp \left(-\frac{1}{2{\sigma }^2(\mathbf{x})}{\left(y-{\mathbf{w}}^{\top }\mathbf{x}\right)}^2\right)$

B) Related

• least squares estimation

C) References