General Linear Model — 2

In our blog General and Generalized Linear Models we covered concepts of General and Generalized Linear Models, their components and difference between them.

In this blog we will cover General Linear model

Regression Model

Types of Relationships

Linear Regression Model

Linear regression is a supervised learning algorithm used when target / dependent variable continues real number.

It establishes relationship between dependent variable Y and one or more independent variable X using best fit line.

Hypothesis: A linear model makes a “hypothesis” about the true nature of the underlying function — that it is linear.

The null hypothesis states that all coefficients in the model are equal to zero. In other words, none of the predictor variables have a statistically significant relationship with the response variable, y.

The alternative hypothesis states that not every coefficient is simultaneously equal to zero.

We express this hypothesis in the univariate/multivariate case as below mentioned equation.

For Simple/univariate Linear Regression

For Multivariate Linear Regression

Matrix Formulation

Linear Regression Model : — Relationship between variables is a linear function.

linear regression formula

Estimating Parameters: Least Squares Method

Least Squares method is a statistical procedure to find the best fit for a set of data points by minimizing the sum of the offsets or residuals of points from the plotted curve.

Scattergram

How to determine which line ‘fits best’?

which line fits best?

Least Squares helps us to determine best fit line.

“Best fit” means difference between actual y values and predicted y values are at minimum.
Least Squares minimizes the Sum of the Squared Differences (SSE)

Computational table and interpretation of coefficient.

Slope (b1) : — The slope indicates the steepness of a line.

Estimated y changes by b1 for each 1unit increase in x.

—e.g, If b1 = 2, then y is expected to increase by 2 for each 1 unit increase in x.

Intercept (b0) :- Intercept indicates the location where it intersects an axis.

• Average value of y when x = 0

—e.g., If b0 = 4, then y is expected to be 4 when x is 0.

Example: —

Coefficient Interpretation Solution

Slope (b1) : — Sales Volume (y) is expected to increase by .7 units for each $1 increase in Advertising (x).

Y-Intercept (b0): — Average value of Sales Volume (y) is -.10 units when Advertising (x) is 0.

— Difficult to explain to marketing manager.
— Expect some sales without advertising

Linear Regression Assumptions

1. Mean of probability distribution of error, ε, is 0.
2. Probability distribution of error has constant variance.
3. Probability distribution of error, ε, is normal.
4. Errors are independent

Random Error Variation

• Variation of actual y from predicted y, y
• Measured by standard error of regression model
  – Sample standard deviation of e : s
• Affects several factors
  - Parameter significance
  - Prediction accuracy

In upcoming blogs we will cover other aspects of linear regression.

Thank you for reading. Links to other blogs: —

General and Generalized Linear Models
The Poisson Distribution
The Geometric and Exponential Distributions
Uniform Distribution
Normal Distribution
Binomial Distribution
10 alternatives for Cloud based Jupyter notebook!!