Least Squares Estimation

Priliminaries A Simple Linear Regression the column space Another Example of Linear Regression 1 In the blog A Simple Linear Regression, squares of the difference between the output of a predictor and the target were used as a loss function in a regression problem. And it could be also written as:
[ ell(hat{mathbf{y}}_i,mathbf{y}_i)=(hat{mathbf{y}}_i-mathbf{y}_i)^T(hat{mathbf{y}}_i-mathbf{y}_i) tag{1} ]
The linear regression model in a matrix form is:
[ y=mathbf{w}^Tmathbf{x}+mathbf{b}tag{2} ]
What we do in this post is analyze the least-squares methods from two different viewpoints