Linear Regression model in Machine Learning

There are many model which could be used to best fit the data.The simplest one is Linear Regression.

In this we fit the data which passes through these points.In liner regression we model the relationship between dependent variable y and independent variable x.
  These relationship are modeled using linear predictor function whose unknown model parameter are estimated from data.

The simple linear regression is given by the following equation:

Here ϵ is the Gaussian noise which is due to randomness of the data

Then plot will be:

Now our main objective is to how to learn the parameter.

We can find the parameter of the  equation of linear regression i.e:

where 

and 

 For multi variable linear regression it is given by the  following equation:

b0 is the intercept and  bj is the slope for the jth variable of the variable Xj.

Now to draw the line we select that one for which which the sum of squared error is as small as possible.

 The sum of squared error is given by:

 For multiple linear regression the hypothesis h(x) is given by:

Here the parameter for this equation is:

To learn this parameter we use Least Mean Square(LMS) algorithm:

• First we make h(x) close to y, for the available training example.

• Then we define the cost function  J(θ) as: 
  
• Then we find θ that minimizes J(θ)

There are various ways of minimizing this cost function to get the

optimal solution.One of which is known as Gradient Descent.

To do this:

         1. First we start with initial guess of θ.

          2. Then we relatively update θ to make J(θ)  smaller until it converges to minima.

We update θ  well follow the following derivation:

Following this update rule the equation will become:

GGiven below is the video in which I have explained about linear regression model of machine learning algorithm:

Hope you have enjoyed reading this article.In next article i will be discussing about the platform on which i will be implementing this algorithm and how to implement it.Till then enjoy learning.