- Introduction
- General Definition
- The tensor product is not the same as the Cartesian product
- Applications

## Introduction

For any vectors $\left\langle x_{1},x_{2}\right\rangle $ of $\mathbb{R}^{2}$ and $\left\langle y_{1},y_{2},y_{3}\right\rangle $ of $\mathbb{R}^{3}$, a product of these two vectors, which is denoted as $\left\langle x_{1},x_{2}\right\rangle \otimes\left\langle y_{1},y_{2},y_{3}\right\rangle $, is defined as the matrix

\[

\left[\begin{array}{ccc}

x_{1}y_{1} & x_{1}y_{2} & x_{1}y_{_{3}}\\

x_{2}y_{1} & x_{2}y_{2} & x_{2}y_{3}

\end{array}\right].

\]

The collection of all the products $\left\langle x_{1},x_{2}\right\rangle \otimes\left\langle y_{1},y_{2},y_{3}\right\rangle $, up to some equivalence, is called the tensor product of $\mathbb{R}^{2}$ and $\mathbb{R}^{3}$ and is denoted by $\mathbb{R}^{2}\otimes\mathbb{R}^{3}$.

## General Definition

For any vector spaces $U,V,W$ over a field $\mathbb{F}$ and any bilinear map $f$ from the Cartesian product $U\times V$ to $W$, and for any basis $B_{U}$ of $U$ and basis $B_{V}$ of $V$, we have this result, which can be checked by expanding the definitions of a basis and of a bilinear map:

**Theorem 1**: If $f$ is onto, the collection of all $f\left(x,y\right)$, with $x$ in $B_{U}$ and $y$ in $B_{V}$, is a basis of $W$.

Here is a definition of the tensor product: the vector space $W$ with the basis consisting of all $f\left(x,y\right)$, with $x$ in $B_{U}$ and $y$ in $B_{V}$, is called the **tensor product** of $U$ and $V$ whenever $f$ is onto, and $W$ is denoted in terms $U$ and $V$ as $U\otimes V$.

## The tensor product is not the same as the Cartesian product

The Cartesian product $U\times V$ of any two vector spaces $U,V$ over the same field is a vector space over that field with addition defined as $\left\langle x_{1},y_{1}\right\rangle +\left\langle x_{2},y_{2}\right\rangle :=\left\langle x_{1}+x_{2},y_{1}+y_{2}\right\rangle $ and scalar multiplication defined as $k\left\langle x_{1},y_{1}\right\rangle :=\left\langle kx_{1},ky_{1}\right\rangle $.

We will show you that the Cartesian product is not the same as the tensor product: we will exhibit two vector spaces whose Cartesian product is not isomorphic to their tensor product.

We take the vector spaces $U,V$ to be the field $\mathbb{F}_{2}$ over itself. The field $\mathbb{F}_{2}$ has two elements $0$ and $1$ with multiplication and addition defined as

\[

\begin{array}{cccccccc}

\times & 0 & 1 & & & + & 0 & 1\\

0 & 0 & 0 & & & 0 & 0 & 1\\

1 & 0 & 1 & & & 1 & 1 & 0

\end{array}.

\]

The addition and scalar multiplication on $U$ and $V$ are exactly as defined above, and the basis of $\mathbb{F}_{2}$ is the singleton $\left\{ 1\right\} $. The elements of the Cartesian product $\mathbb{F}_{2}\times\mathbb{F}_{2}$ are the collection of pairs $\left\{ \left\langle 0,0\right\rangle ,\left\langle 0,1\right\rangle ,\left\langle 1,0\right\rangle ,\left\langle 1,1\right\rangle \right\} $.

The subcollection $\left\{ \left\langle 0,1\right\rangle ,\left\langle 1,0\right\rangle \right\} $ spans $\mathbb{F}_{2}\times\mathbb{F}_{2}$ and is linearly independent, so it is a basis. For the tensor product $\mathbb{F}_{2}\otimes\mathbb{F}_{2}$, recall from the above definition that there is an onto bilinear map $f$ from $\mathbb{F}_{2}\times\mathbb{F}_{2}$ to $\mathbb{F}_{2}\otimes\mathbb{F}_{2}$, so by Theorem 1, the singleton $\left\{ f\left(1,1\right)\right\} $ is a basis of the tensor product. Since $\mathbb{F}_{2}\times\mathbb{F}_{2}$ has a basis with $2$ elements and $\mathbb{F}_{2}\otimes\mathbb{F}_{2}$ has a basis with $1$ element, they cannot be isomorphic.

### A jump to categories

The Cartesian product and the tensor product can be seen as bifunctors from $\mathbb{F}- \mathbf{Vect}\times\mathbb{F}-\mathbf{Vect}$ to $\mathbb{F}-\mathbf{Vect}$, where $\mathbb{F}-\mathbf{Vect}$ is the category of vector spaces over a common field $\mathbb{F}$. Since we have shown above that $\mathbb{F}_{2}\times\mathbb{F}_{2}$ is not isomorphic to $\mathbb{F}_{2}\otimes\mathbb{F}_{2}$, we have the following:

**Corollary 2**: As bifunctors, the Cartesian product and the tensor product are not naturally isomorphic.

**Remark**: The tensor product can be defined for other structures, such as modules, topological spaces, graphs, and matrices.

## Applications

#### TensorFlow

Tensors are elements of a tensor product of vector spaces. TensorFlow is a software library, which performs computations on tensors. TensorFlow is used in machine learning applications, such as building neural networks.

#### Artificial Intelligence

In this paper on natural language processing, the tensor product is used to represent strings of words in a vector space.

#### Quantum Optics

The displacement operator on several modes can be decomposed as a tensor product. For instance, see this paper, section III.

———————————————————————————————————————————————————————–