Vector space

A vector space, also known as a linear space, is an abstract mathematical construct with many important applications in the natural sciences, in particular in physics and numerous areas of mathematics. Some vector spaces make sense somewhat intuitively, such as the space of 2D vectors in standard Euclidean plane, and the language that we use when talking about these intuitive spaces has been taken to describe the more abstract notion as well. For example, we know how to add vectors and multiply them by real numbers (scalars) in $\mathbb {R} ^{3}$, and these notions of vector addition and scalar multiplication are defined in a more general sense (as we will see below).

Vector spaces are important because many different mathematical objects that at first glance seem unrelated in fact share a common structure. By defining this structure and proving things about it in general, we are then able to apply these results to each specific case without having to re-prove them each time. Besides vectors in $\mathbb {R} ^{3}$ that are relatively easy to visualize, we can make a vector space out of $\mathbb {R} ^{n}$ for any natural number n; or the complex plane or powers of it, $\mathbb {C} ^{n}$; or polynomials of degree n.

No matter what vector space you have to work with though, it is often useful to keep a picture of either 2D or 3D space in mind. This helps when thinking of things such as orthogonal polynomials or matrices.

Definition

A vector space $V$ over a field $F$ is a set that satisfies certain axioms (see below) and which is equipped with two operations, vector addition and scalar multiplication. Vector addition is defined as a map

$+:\quad V\times V\to V$

that takes the ordered pair $({\vec {u}},{\vec {v}})\in V\times V$ to the vector ${\vec {u}}+{\vec {v}}$. Here $\times$ represents the Cartesian product between sets. Scalar multiplication is defined in a similar way, as a map

$\cdot :\quad F\times V\to V$

that takes the ordered pair $(a,{\vec {u}})\in F\times V$ to the vector $a\cdot {\vec {u}}$. Note that frequently the dot representing scalar multiplication is omitted, the result being written simply as $a{\vec {u}}$ instead. This is especially common when an inner product will also be defined on the vector space, with the dot then representing the inner product between two vectors. It is important to keep in mind the distinction between scalar multiplication, which multiplies one vector by a scalar, and an inner or scalar product, that combined two vectors to yield a scalar.

Axioms of a vector space

Let $V$ be a set, ${\vec {u}}$, ${\vec {v}}$, and ${\vec {w}}$ elements of that set, and $a$ and $b$ scalar elements of a field, $F$. Then $V$ is a vector space if the following axioms hold true for all choices of ${\vec {u}},\ {\vec {v}},\ a,\ b$

1. $V$ is closed under addition

The vector ${\vec {u}}+{\vec {v}}$ is also an element of $V$. This is automatically satisfied when the addition operation is defined as being injective as it was above. Care must be taken however if $V$ is a subset of some larger set $W$ and $+:\,\,V\times V\to W$, as is often the case when looking at subspaces.

2. Addition is commutative

The order in which two vectors are added does not affect the result, ${\vec {u}}+{\vec {v}}={\vec {v}}+{\vec {u}}$.

3. Addition is associative

${\vec {u}}+({\vec {v}}+{\vec {w}})=({\vec {u}}+{\vec {v}})+{\vec {w}}$. This means that even though addition is strictly defined as a binary operation, the object ${\vec {u}}+{\vec {v}}+{\vec {w}}$ is well defined.

4. An additive identity exists in $V$

Labeled ${\vec {0}}$, the additive identity or zero vector satisfies ${\vec {0}}+{\vec {u}}={\vec {u}}+{\vec {0}}={\vec {u}}$.

5. The additive inverse exists in $V$

A vector $-{\vec {u}}$ can be found such that $-{\vec {u}}+{\vec {u}}={\vec {u}}+(-{\vec {u}})={\vec {0}}$.

6. $V$ is closed under scalar multiplication

The vector $a{\vec {u}}$ is itself an element of $V$.

7. Scalar multiplication is distributive over addition in $F$

$(a+b){\vec {u}}=a{\vec {u}}+b{\vec {u}}$. It is important to note that the addition occurring on the left-hand side of this equality is a 'different operation' from the addition on the right-hand side. While the latter is vector addition as defined above, the former is the addition operation defined on the field $F$.

8. Vector addition is distributive over scalar multiplication

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle a(\vec{u}+\vec{v})=a\vec{u}+a\vec{v}} . In this case vector addition takes place on both sides of the equality.

9. Scalar multiplication is associative

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle a(b\vec{u})=(ab)\vec{u}} . This means that the algebraic structure of the underlying field Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle F} is preserved. Note that the left-hand side of this equality contains two subsequent applications of the scalar multiplication defined above, while the right-hand side contains one scalar multiplication as defined in Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle F} (that of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle ab} ), followed by scalar multiplication with the vector Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{u}} .

10. The multiplicative identity of $F$ provides a scalar multiplicative identity

$1{\vec {u}}={\vec {u}}$, where $1$ is the multiplicative identity of the field $F$.

Properties 1 - 5 state that a vector space is an Abelian group with addition as group operation.

These axioms can be expressed concisely in mathematical notation as follows:

$\forall {\vec {u}},{\vec {v}},{\vec {w}}\in V,\ \forall a,b\in F,$

1. ${\vec {u}}+{\vec {v}}\in V$
2. ${\vec {u}}+{\vec {v}}={\vec {v}}+{\vec {u}}$
3. ${\vec {u}}+({\vec {v}}+{\vec {w}})=({\vec {u}}+{\vec {v}})+{\vec {w}}$
4. $\exists {\vec {0}}\in V:{\vec {0}}+{\vec {u}}={\vec {u}}+{\vec {0}}={\vec {u}}$
5. $\exists \ -\!{\vec {u}}\in V:-{\vec {u}}+{\vec {u}}={\vec {u}}+(-{\vec {u}})={\vec {0}}$
6. $a{\vec {u}}\in V$
7. $(a+b){\vec {u}}=a{\vec {u}}+b{\vec {u}}$
8. $a({\vec {u}}+{\vec {v}})=a{\vec {u}}+a{\vec {v}}$
9. $a(b{\vec {u}})=(ab){\vec {u}}$
10. $1{\vec {u}}={\vec {u}}$

Some important theorems

Linear dependence

A system of p ( ≥ 1 ) vectors ${\vec {u}}_{1},\dots ,{\vec {u}}_{p}$ of a vector space V is called linearly dependent if there exist coefficients (elements in F ) a₁, ..., a_p not all zero, such that the linear combination is the zero vector in V,

$\sum _{\nu }\,a_{\nu }\,{\vec {u}}_{\nu }={\vec {0}}\in V.$

Otherwise, the vectors ${\vec {u}}_{1},\dots ,{\vec {u}}_{p}$ are called linearly independent. A single vector not equal to the zero vector is obviously linearly independent.

If all a₁, ..., a_p are zero (in F ) then   $\sum _{\nu }\,a_{\nu }\,{\vec {u}}_{\nu }={\vec {0}}.$   If the set ${\vec {u}}_{1},\dots ,{\vec {u}}_{p}$ is linearly independent then the relation   $\sum _{\nu }\,a_{\nu }\,{\vec {u}}_{\nu }={\vec {0}}$ implies that all a₁, ..., a_p are zero. Hence a set of p vectors in V is linearly independent if

$\sum _{\nu }\,a_{\nu }\,{\vec {u}}_{\nu }={\vec {0}}\quad \Longleftrightarrow \quad a_{\nu }=0,\quad {\hbox{for}}\quad \nu =1,\ldots ,p.$

Every set of vectors containing the zero vector is linearly dependent.

A system of linearly independent vectors ${\vec {u}}_{1},\dots ,{\vec {u}}_{p}$ remains linearly independent if some vectors are omitted from the system. For, let a subset of the first q vectors ${\vec {u}}_{1},\dots ,{\vec {u}}_{q}$, with q < p, be linearly dependent then one or more coefficients not equal to zero can be found while the following is true

$a_{1}{\vec {u}}_{1}+\cdots +a_{q}{\vec {u}}_{q}={\vec {0}}.$

Add to the left- and right-hand side of this expression   ${\vec {0}}=0\cdot {\vec {u}}_{q+1}+\cdots +0\cdot {\vec {u}}_{p}$   and we get a contradiction.

Dimension

In general there are infinitely many linearly independent vectors in a vector space. When the maximum number of linearly independent vectors is finite, say n, the vector space is called of finite dimension n. Otherwise the space is called infinite-dimensional. If V′ is a linear subspace of the n-dimensional space V (all elements of V′ belong simultaneously to V ), and V′ contains a set B of m linearly independent vectors then m < n, because B belongs to the n-dimensional space V. It follows that m is finite and that all subspaces of finite-dimensional spaces are finite-dimensional. If m is the maximum number of linearly independent vectors in V′ then this subspace is of dimension m < n. For finite n it can be shown that V′ coincides with V (is an "improper" subspace) if and only if n = m.

Examples of vector spaces

Except for the Euclidean plane, the best known vector space is the space $\mathbb {R} ^{n}$.  For integral finite n > 0 this space consists of columns (stacks) of n real numbers. In order to make the discussion concrete we consider the case n = 4. It will be clear how the rules apply to general finite n.

Addition

${\begin{pmatrix}x_{1}\\x_{2}\\x_{3}\\x_{4}\\\end{pmatrix}}+{\begin{pmatrix}y_{1}\\y_{2}\\y_{3}\\y_{4}\\\end{pmatrix}}={\begin{pmatrix}x_{1}+y_{1}\\x_{2}+y_{2}\\x_{3}+y_{3}\\x_{4}+y_{4}\\\end{pmatrix}}$

Because x_k and y_k are real numbers, x_k+y_k is a well-defined real number.

Negative vector

${\begin{pmatrix}x_{1}\\x_{2}\\x_{3}\\x_{4}\\\end{pmatrix}}+{\begin{pmatrix}-x_{1}\\-x_{2}\\-x_{3}\\-x_{4}\\\end{pmatrix}}={\begin{pmatrix}x_{1}-x_{1}\\x_{2}-x_{2}\\x_{3}-x_{3}\\x_{4}-x_{4}\\\end{pmatrix}}={\begin{pmatrix}0\\0\\0\\0\\\end{pmatrix}}$

Zero vector

${\begin{pmatrix}x_{1}\\x_{2}\\x_{3}\\x_{4}\\\end{pmatrix}}+{\begin{pmatrix}0\\0\\0\\0\\\end{pmatrix}}={\begin{pmatrix}x_{1}\\x_{2}\\x_{3}\\x_{4}\\\end{pmatrix}}$

Multiplication by real number

$a{\begin{pmatrix}x_{1}\\x_{2}\\x_{3}\\x_{4}\\\end{pmatrix}}={\begin{pmatrix}a\,x_{1}\\a\,x_{2}\\a\,x_{3}\\a\,x_{4}\\\end{pmatrix}}$

Because a and x_k are real numbers, a x_k is well-defined and real.

The reader may easily convince him/herself, using the known properties of real numbers, that these columns of real numbers satisfy the postulates of a vector space. Its dimension is at least 4, because the following 4 vectors are linearly independent,

${\begin{pmatrix}1\\0\\0\\0\\\end{pmatrix}},\quad {\begin{pmatrix}0\\1\\0\\0\\\end{pmatrix}},\quad {\begin{pmatrix}0\\0\\1\\0\\\end{pmatrix}},\quad {\begin{pmatrix}0\\0\\0\\1\\\end{pmatrix}}.\qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (1)$

Indeed, assume that one or more of the coefficients (real numbers) a_k is not equal to zero, then the equation

$a_{1}{\begin{pmatrix}1\\0\\0\\0\\\end{pmatrix}}+a_{2}{\begin{pmatrix}0\\1\\0\\0\\\end{pmatrix}}+a_{3}{\begin{pmatrix}0\\0\\1\\0\\\end{pmatrix}}+a_{4}{\begin{pmatrix}0\\0\\0\\1\\\end{pmatrix}}={\begin{pmatrix}a_{1}\\a_{2}\\a_{3}\\a_{4}\\\end{pmatrix}}={\begin{pmatrix}0\\0\\0\\0\\\end{pmatrix}}.$

leads to all four a′s are zero (two vectors are equal if and only if their corresponding elements are equal). This is in contradiction to the assumption that one or more of the coefficients a_k is not equal to zero.

The set (1) is maximally linearly independent because any non-zero vector can be expressed in the four vectors,

${\begin{pmatrix}x_{1}\\x_{2}\\x_{3}\\x_{4}\\\end{pmatrix}}=x_{1}{\begin{pmatrix}1\\0\\0\\0\\\end{pmatrix}}+x_{2}{\begin{pmatrix}0\\1\\0\\0\\\end{pmatrix}}+x_{3}{\begin{pmatrix}0\\0\\1\\0\\\end{pmatrix}}+x_{4}{\begin{pmatrix}0\\0\\0\\1\\\end{pmatrix}}\quad \Longrightarrow \quad -1{\begin{pmatrix}x_{1}\\x_{2}\\x_{3}\\x_{4}\\\end{pmatrix}}+x_{1}{\begin{pmatrix}1\\0\\0\\0\\\end{pmatrix}}+x_{2}{\begin{pmatrix}0\\1\\0\\0\\\end{pmatrix}}+x_{3}{\begin{pmatrix}0\\0\\1\\0\\\end{pmatrix}}+x_{4}{\begin{pmatrix}0\\0\\0\\1\\\end{pmatrix}}={\begin{pmatrix}0\\0\\0\\0\\\end{pmatrix}}.$

The equation on the right is a valid equation between five vectors that do not have a prefactor zero and yet give the zero vector. Hence it is not possible to find a fifth vector linearly independent of the vectors (1): any five vectors form a linearly dependent set. In other words, the four vectors in Eq. (1) form a basis of the vector space $\mathbb {R} ^{n}$.