Linear Algebra tutorial: Vector Norm

Vector Norm

Based on Pythagorean Theorem, the vector from the origin to the point (3, 4) in 2D Euclidean plane has length of $ \sqrt{3^{2}+4^{2}}=\sqrt{25}=5 $ and the vector from the origin to the point $ (a,b)$ has length $ \sqrt{a^{2}+b^{2}} $. The length of a vector with two elements is the square root of the sum of each element squared.

The magnitude of a vector is sometimes called the length of a vector, or norm of a vector. Basically, norm of a vector is a measure of distance, symbolized by double vertical bar $ \left \| \mathbf{a} \right \| $
The magnitude of a vector can be extended to $ n $ dimensions. A vector a with $ n $ elements has length
$$ \left \| \mathbf{a} \right \| = \sqrt{a_{1}^{2}+a_{2}^{2}+...+a_{n}^{2}} $$

The vector length is called Euclidean length or Euclidean norm. Mathematician often used term norm instead of length. Vector norm is defined as any function that associated a scalar with a vector and obeys the three rules below

Norm of a vector is always positive or zero $ \left \| \mathbf{a} \right \| \geqslant 0 $. The norm of a vector is zero if and only if the vector is a zero vector $ \mathbf{a} = \mathbf{0} $.
A scalar multiple to a norm is equal to the product of the absolute value of the scalar and the norm$ \left \| k\mathbf{a} \right \|=\left | k \right |\left \| \mathbf{a} \right \| $.
Norm of a vector obeys triangular inequality that the norm of a sum of two vectors is less than or equal to the sum of the norms $ \left \| \mathbf{a} + \mathbf{b}\right \| \leqslant \left \| \mathbf{a} \right \| + \left \| \mathbf{b} \right \| $.

There are many common norms:

1-norm is defined by the sum of absolute value of the vector elements
$ \left \| \mathbf{a} \right \|_{1} = \left | a_{1} \right | + \left | a_{2} \right | + ... + \left | a_{n} \right | $ .
2-norm is the most often used vector norm, sometimes called Euclidean norm. When the subscript index of the vector norm is not specified, you may think that it is a Euclidean norm $ \left \| \mathbf{a} \right \| = \left \| \mathbf{a} \right \|_{2} = \sqrt{a_{1}^{2}+a_{2}^{2}+...+a_{n}^{2}} $ .
p-norm is sometimes called Minskowski norm is defined as $ \left \| \mathbf{a} \right \|_{p} = \sqrt[p]{a_{1}^{p}+a_{2}^{p}+...+a_{n}^{p}} $ . p-norm is generalized norm with a parameter $ p$ .
max-norm is also called Chebyshev norm is the largest absolute element in the vector $ \left \| \mathbf{a} \right \|_{\infty} = max[\left | a_{1} \right | , \left | a_{2} \right | , ... , \left | a_{n} \right | ] $

Use the interactive program below to experiment with your own vector input. The program will give you the norm of vector for p=1, 2, 3 and max. The vector input will be redrawn to give you feedback on what you type. Click Random Example button to generate random vector.

Properties

Some important properties of vector norm are

Square of Euclidean norm is equal to the sum of square $ \left \| \mathbf{a} \right \|^{2} = a_{1}^{2} + a_{2}^{2} + ... + a_{n}^{2} $.
Pythagorean Theorem is hold if and only if the two vectors are orthogonal $ \left \| \mathbf{a}-\mathbf{b} \right \|^{2} = \left \| \mathbf{a} \right \|^{2}+\left \| \mathbf{b} \right \|^{2}, \mathbf{a} \perp \mathbf{b} $.
The law of cosine $ \left \| \mathbf{a}-\mathbf{b} \right \|^{2} = \left \| \mathbf{a} \right \|^{2}+\left \| \mathbf{b} \right \|^{2}-2\left \| \mathbf{a} \right \|\left \| \mathbf{b} \right \|cos\phi $
Norm of the dot product of two vectors is equal to the product of their norms $ \left \| \mathbf{a}^{T}\mathbf{b} \right \| = \left \| \mathbf{a} \right \|\left \| \mathbf{b} \right \| $.
Relationship to vector inner products
- Square of Euclidean norm of a vector is equal to the inner product to itself $ \left \| \mathbf{a} \right \|^{2} = \mathbf{a}^{T}\mathbf{a} $
- $ \mathbf{a}^{T}\mathbf{b}=\left \| \mathbf{a} \right \|\left \| \mathbf{b} \right \|cos\phi $ where, $ \phi $ is the angle between the two vectors
- $ \mathbf{a}^{T}\mathbf{b} = \frac{1}{4} (\left \| \mathbf{a}+\mathbf{b} \right \|^{2}-\left \| \mathbf{a}-\mathbf{b} \right \|^{2}) $
- Norm of addition or subtraction follow the law of cosine $ \left \| \mathbf{a} \pm \mathbf{b} \right \|^{2} = \left \| \mathbf{a} \right \|^{2} + \left \| \mathbf{b} \right \|^{2} \pm 2 \mathbf{a}^{T} \mathbf{b} $
Addition of two square of norm of vectors follow parallelogram law $ \left \| \mathbf{a} \right \|^{2} + \left \| \mathbf{b} \right \|^{2} =\frac{1}{2}(\left \| \mathbf{a} + \mathbf{b} \right \|^{2} + \left \| \mathbf{a} - \mathbf{b} \right \|^{2}) $
p-norm is greater than the max-norm but less than $ n^{\frac{1}{p}} $ times the max-norm, that is $ 1\leqslant \frac{\left \| \mathbf{a} \right \|_{p}}{\left \| \mathbf{a} \right \|_{\infty }}\leq n^{\frac{1}{p}} $ .
The norm ratio satisfies the inequality $ 1\leqslant \frac{\left \| \mathbf{a} \right \|_{p}}{\left \| \mathbf{a} \right \|_{q }}\leq n^{\frac{q-p}{pq}} $ . As $ q $ tends to infinity, the $ \frac{q-p}{pq} $ approaches $ \frac{1}{p} $ and $ \left \| \mathbf{a} \right \|_{p} $ approaches $ \left \| \mathbf{a} \right \|_{\infty} $.
Cauchy-Schwartz inequality stated that the absolute value of vector dot product is always less than or equal to the product of their norms $ \left | \mathbf{a}^{T}\mathbf{b} \right |\leq \left \| \mathbf{a} \right \|\left \| \mathbf{b} \right \| $ . The equality $ \left | \mathbf{a}^{T}\mathbf{b} \right | = \left \| \mathbf{a} \right \|\left \| \mathbf{b} \right \| $ holds if and only if the vectors are linearly dependent.
Relationship of norm of cross product and dot product is $ \left \| \mathbf{a} \times \mathbf{b} \right \|^{2} + \left \| \mathbf{a}\cdot \mathbf{b} \right \|^{2}=\left \| \mathbf{a} \right \|^{2}\left \| \mathbf{b} \right \|^{2} $ .

Rate this tutorial or give your comments about this tutorial

This tutorial is copyrighted.

Preferable reference for this tutorial is

Teknomo, Kardi (2011) Linear Algebra tutorial. https:\\people.revoledu.com\kardi\tutorial\LinearAlgebra\