Probstat/notes/regression

This is part of probstat.

In this section, we shall discuss linear regression. We shall focus on one-variable linear regression.

Model

We consider two variables $X$ and $Y$ where $Y$ is a function of $X$. We refer to $X$ as independent or input variable, and $Y$ as a dependent variable. We consider linear relationship between independent variable and dependent variable. We assume that there exist hidden variables $\alpha$ and $\beta$ such that

$Y=\alpha +\beta \cdot X+e,$

where $e$ is a random error. We further assume that the error is unbiased, i.e., $E[e]=0$ and is independent of $X$.

Input: As an input to the regression process, we are given a set of $n$ data points: $(x_{1},y_{1}),(x_{2},y_{2}),\ldots ,(x_{n},y_{n})$ generated from the previous equation.

Goal: We want to estimate $\alpha$ and $\beta$.