Short blog on First order and Second order — Calculus
Machine learning uses derivatives in optimization problems. Optimization algorithms like gradient descent, derivatives represent a slope on a curve, they can be used to find maxima and minima of functions, when the slope, is zero.
Easiest explanation of a derivative is just simply the slope at any given point “x”. We have already seen the same in mathematics. a derivative is typically notated as dy/dx.
Recommended article on derivative : — What Does f’ Say About f?
First-Order Derivative
The first order derivative of a function represents the rate of change of one variable with respect to another variable.
For example, in Physics we define the velocity of a body as the rate of change of the location of the body with respect to time.
First derivative of a given function gives you the slope of that function at any given point. As well as this, it can show you local extrema (maximums and minimums) when you graph it. These are your 0’s on the graph.
Second-Order Derivative
The second order derivatives are used to get an idea of the shape of the graph for the given function. Whether it will be upwards to downwards. It is based on concavity. The concavity of the given graph function is classified into two types: — Concave up and Concave down.
The second derivative is the first derivative of the first derivative. In general the n+1st derivative (with respect to x) of a (differentiable) function of x, is the first derivative of the nth derivative. This goes all the way ‘down’ to the zeroth derivative which is the function itself.
The second derivative is the slope of the first derivative.
Normally, the second derivative of a given function links to the concavity of the graph. If the second-order derivative value is positive, then the graph of a function is upwardly concave. If the second-order derivative value is negative, then the graph of a function is downwardly open.
We already know that second derivative of a function determines the local maximum (maxima) or minimum (minima), inflexion point values. These can be identified with the help of below conditions:
- If f”(x) < 0, then the function f(x) has a local maximum at x.
- If f”(x) > 0, then the function f(x) has a local minimum at x.
- If f”(x) = 0, then it is not possible to conclude anything about the point x, a possible inflexion point.
Relation between first and second derivative
The relation between 1st and 2nd derivative is the same as between a function and its 1st derivative. The first derivative is the rate of change. The second derivative is the rate of change of the rate of change.
A common practical application is if you measure distance/time, the first derivative will be velocity (speed) and the second the acceleration.
You can test whether your extreme found in the first derivative is a maximum or minimum by plugging in the x value into the equation. If the subsequent y value is less than 0, then you have a maximum. If your value is greater than 0, then you have a minimum. As well as this, the zeroes of this function show inflection points for the concavity (whether the graph is facing up or down). You can test these points by plugging in a value higher and lower than the 0. If there is a sign change, then it is an inflection point.
Constant Acceleration Motion : —
Picked some data from hyper physics for constant acceleration motion.
For Above mentioned screenshot: —
- The first one is how position is changing over time. So, at periodic points, the position is measured. How far something has moved.
- The second one is the measure of speed (because we are not considering direction), how much the position changes per unit time period. This is the first derivative.
- The third one is the acceleration, how much the velocity changes per unit time period. This is the second derivative.
Thank you for reading. Links to other blogs: —
Statistical Inference 2 — Hypothesis Testing
Statistical Inference
Bayesian Generalized Linear Model (Bayesian GLM) — 2
Central Limit Theorem — Statistics
General Linear Model — 2
General and Generalized Linear Models
10 alternatives for Cloud based Jupyter notebook!!
